python: Don't abuse hex()

python: Don't abuse hex()

The hex() builtin returns a string containing the hexa-decimal
representation of an integer.

When the argument is not an integer, then the function calls that
object's __hex__() method, if one is defined. That method is supposed to
return a string.

While that's not explicitly documented, that string is supposed to be a
valid hexa-decimal representation for a number. Python 2 doesn't enforce
this though, which is why we got away with returning things like
'NIR_TRUE' which are not numbers.

In Python 3, the hex() builtin instead calls an object's __index__()
method, which itself must return an integer. That integer is then
automatically converted to a string with its hexa-decimal representation
by the rest of the hex() function.

As a result, we really can't make this compatible with Python 3 as it
is.

The solution is to stop using the hex() builtin, and instead use a hex()
object method, which can return whatever we want, in Python 2 and 3.

Signed-off-by: Mathieu Bridon <bochecha@daitauha.fr>
Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>