Friday, February 24, 2012

Data types - using integer as numerator in a percentage calculation

I know I'm probably going to get rapped in the mouth for my stupidity but,
in SQL Server 2000
Price is money(8), Qty Is int(4)
Discount Is int(4) - the program that it feeds into doesn't accept fractions
of a discount
The statement:
SELECT Items.Price, Items.discount,
(Items.Price - (Items.Price * (Items.discount / 100))) *
Items.Qty AS Extension
FROM Items
Returns
29.9900 10 29.98000
Heck, even
SELECT (10/100)
reurns zero
SELECT CAST((10/100) AS Real)
returns 0.0
What am I doing wrong?
JulianOh, please don't tell me that all I had to do was put a ".0" at the end of
the "100" ...
julian
"stjulian" <anonymous@.discussions.microsoft.com> wrote in message
news:ef%23EbupJGHA.1424@.TK2MSFTNGP12.phx.gbl...
>I know I'm probably going to get rapped in the mouth for my stupidity but,
> in SQL Server 2000
> Price is money(8), Qty Is int(4)
> Discount Is int(4) - the program that it feeds into doesn't accept
> fractions
> of a discount
> The statement:
> SELECT Items.Price, Items.discount,
> (Items.Price - (Items.Price * (Items.discount / 100))) *
> Items.Qty AS Extension
> FROM Items
> Returns
> 29.9900 10 29.98000
>
>
> Heck, even
> SELECT (10/100)
> reurns zero
>
> SELECT CAST((10/100) AS Real)
> returns 0.0
>
> What am I doing wrong?
> Julian
>
>
>|||Julian, You need to CAST early, either implicitly or explicitly
(Items.Price - (Items.Price * (Items.discount / 100.0)))
Adding the .0 to the end of 100 will cause it to be treated as float. Or
you can CAST you number as DECIMAL(5,2) or whatever works for you.
RLF
"stjulian" <anonymous@.discussions.microsoft.com> wrote in message
news:ef%23EbupJGHA.1424@.TK2MSFTNGP12.phx.gbl...
>I know I'm probably going to get rapped in the mouth for my stupidity but,
> in SQL Server 2000
> Price is money(8), Qty Is int(4)
> Discount Is int(4) - the program that it feeds into doesn't accept
> fractions
> of a discount
> The statement:
> SELECT Items.Price, Items.discount,
> (Items.Price - (Items.Price * (Items.discount / 100))) *
> Items.Qty AS Extension
> FROM Items
> Returns
> 29.9900 10 29.98000
>
>
> Heck, even
> SELECT (10/100)
> reurns zero
>
> SELECT CAST((10/100) AS Real)
> returns 0.0
>
> What am I doing wrong?
> Julian
>
>
>|||Ok then. I won't tell you.
"stjulian" <anonymous@.discussions.microsoft.com> wrote in message
news:%239vjmwpJGHA.1676@.TK2MSFTNGP09.phx.gbl...
> Oh, please don't tell me that all I had to do was put a ".0" at the end of
> the "100" ...
> julian
>
> "stjulian" <anonymous@.discussions.microsoft.com> wrote in message
> news:ef%23EbupJGHA.1424@.TK2MSFTNGP12.phx.gbl...
>|||> SELECT (10/100)
> reurns zero
This is called integer math.
http://www.aspfaq.com/2483|||I'm a dunce.
<8(
julian
"stjulian" <anonymous@.discussions.microsoft.com> wrote in message
news:%239vjmwpJGHA.1676@.TK2MSFTNGP09.phx.gbl...
> Oh, please don't tell me that all I had to do was put a ".0" at the end of
> the "100" ...
> julian
>
> "stjulian" <anonymous@.discussions.microsoft.com> wrote in message
> news:ef%23EbupJGHA.1424@.TK2MSFTNGP12.phx.gbl...
>

No comments:

Post a Comment