"it's not a bug" is a really bizarre response.

"we know" is kind of ok, but the status of bug-hood is defined between coders and users, not solely by coders I think, and this breaks the POLA severely: People who depend on JQ don't expect this.

    >> x = '{"id":675127116845989888,"id_str":"675127116845989888"}'
    <- "{\"id\":675127116845989888,\"id_str\":\"675127116845989888\"}"
    >> JSON.parse(x)
    <- Object { id: 675127116845989900, id_str: "675127116845989888" }

 

That is the problem with JSON. XML worked much better for integers. And if it is typed with XML schema or XPath, XML has an xs:unsignedLong integer type.

I wrote my own command line XPath JSON-query tool and used bigdecimals for JSON numbers. If you do not do much math, bigdecimals are probably even faster than using floats, and much easier to implement. Converting a string to a double float is extraordinary complex. I think it is one of the most difficult tasks in computing.

Unfortunately, now the W3C made a new XPath standard that requires JSON parsers to use double for numbers, so I changed my tool to use doubles as well. Now I am struggling with the string<->float conversion. The conversion in the standard library does not work properly. I just looked at another conversion library. 4000 lines of code, and after a 2 hour investigation, it turns out, it also does not work properly.

For whatever it's worth, on a somewhat-current Linux Mint, with the test from https://github.com/stedolan/jq/issues/1387:

System jq:

    $ jq --version
    jq-1.6
    $ echo '{"number":288230376151711744}' | jq '.number'
    288230376151711740

Fresh compile from source according to the build instructions at https://github.com/stedolan/jq:

    $ ./configure --with-oniguruma=builtin && make -j8
    $ ./jq --version
    jq-1.6-137-gd18b2d0-dirty
    $ echo '{"number":288230376151711744}' | ./jq '.number'
    288230376151711744

Alternatively:

    $ ./configure --with-oniguruma=builtin --enable-decnum=no && make -j8
    $ echo '{"number":288230376151711744}' | ./jq '.number'
    288230376151711740

So the basic bug is fixed, jq has included a bignum library for > 2 years. I don't know if Mint (and thus presumably Ubuntu, and thus possibly Debian) includes an older version of jq or sets nonstandard user-unfriendly flags on purpose, but I'm somewhat underwhelmed in either case.

Ran across this today and spent a good 20 to 25 minutes trying to debug why my own code was failing when it was jq that was rounding an id in the JSON response.

Submitting this to let others that use jq beware of this :/

I really like the idea of just treating numbers as a higher level data type instead of ints, floats, signed, unsigned, or whatever. I wish most high level languages kept floats as an implementation detail, and defaulted to decimals.

Though I don't see a way around it with a Serialization format like JSON that is meant to work across languages. I've done limited work with static languages, but from what I remember it would be a nightmare to have a variable that could be one of many types. I'm thinking overloading, or interfaces, or something. Could someone familiar with like Go or C#, or whatever explain how they would handle that?

Edit:

Writing that reminded me of something. If you are using a 64-bit unsigned integer, and you need to convert it to JSON for public use, please do not just give a object with a high and low value in hex. If you decide to do this anyway, but also ship an official Python SDK, just do the conversion in the SDK. I'm looking at you F5.

I should have never needed to write this function to read memory usage stats, but it was kind of fun figuring it out.

    def ulong64_to_int(ulong64):
        high = ulong64.get('high')
        low = ulong64.get('low')
        return int('{0:032b}{1:032b}'.format(high & 0xffffffff, low & 0xffffffff), 2)

Also note that there is another response with a bit more details in it in a different (not closed) ticket from 2018:

https://github.com/stedolan/jq/issues/1741#issuecomment-4306...

Even processors don't always actually have the capability of doing math on 64 bit numbers, in general, because they're nonsensically large.