Reading a .raw file and display as hex

[random_1]

Hi. I have a .raw file and I want to display it as hex. why? because i want to try and replicate what 010 Editor can do which is read any sort of file and display in whatever format user desires (plus it can be used to edit and stuff) but I'm focusing on .raw files only and to display in hex only.

i sort of understand that binascii.hexlify() returns the hexadecimal representation of the binary data (.raw files are binary data right?) but when i do :

content = open(fileName, "rb")
byte = content.read(1)
hexadecimal = binascii.hexlify(byte)
self.file_content.setText(hexadecimal)

I get : TypeError: setText(self, str): argument 1 has unexpected type 'bytes' , what does that mean? which argument? why doesn't it "return hexadecimal....."?

just to clarify, Im using gui to display the data so self.file_content.setText() is a readonly text edit box. And im using Windows 8.1/Anaconda

[Gribouillis]

In python 3, the value returned by binascii.hexlify() is a bytes instance instead of a str instance. As setText() is expecting a str instance, you need to convert hexadecimal to str. This is done by

hexa_str = hexadecimal.decode('utf-8')

[random_1]

Ah. okay. thank you Griboullis. how would i display all data? I mean the file im using displays 49 49 2A 00 08 ....etc, like the full thing. but I only have the first 49 displayed.

[Gribouillis]

but I only have the first 49 displayed.

It is because of the byte = content.read(1): you are only reading one byte. Use regular python code:

with open(fileName, "rb") as infile:
    content = infile.read()
hex_bytes = binascii.hexlify(content)
hex_str = hex_bytes.decode('utf-8')
self.file_content.setText(hex_str)

[random_1]

ohhh.. I though 1 was like a default ot something. so 1 literally means read 1 byte? This is good for a couple of thousand lines but what about a 100 thousand lines? is there a python function/module for this?

[random_1]

I read that using .readline() and for loop is better for huge files so i tried that but it only read the last hundred lines of the file.

with open(fileName, "rb") as infile:
     for lines in infile:
         content = infile.readline()

[Gribouillis]

Don't use readline() with binary files, because they are not organized in lines. If this is a big file, invoke read() with a size argument to read iteratively chunks of binary data.

with open(filename, 'rb') as infile:
    while True:
        data = infile.read(1024)
        if not data:
            break # we have reached the end of the file
        # do something with the bytes read
        print(hexlify(data).decode('utf-8'))

[random_1]

hmm. it only displayes a few dozen lines of the end of the file. so then i tried:

with open(fileName, "rb") as infile:
    content = infile.read(1024)

it read the first 56 lines of the file which i guess is because of 1024. so read(size) is good, now i gotta figure out how to read the entire thing instead of it stopping at a few dozen lines. so increased to 4096 like:

content = infile.read(4096)

it read more than 1024 AND from the beginning of the file but of course not the whole file. I read that 4096 is the max number that can be used. if i use it with whilethe app crashes (not responding), idk why though.

of course i could do like size = 4096 content = infile.read(size * 4), which would read more but then 4 would need to be increased to a bigger number but thats seems totally wrong/redundant coz obvs in RL that would be absurd use. so now im stuck coz while isn't cooperating.

the file that i am currently working with (to test) is only 16.2 mb but the exact file size i will be using is 8gb.

[Gribouillis]

hmm. it only displayes a few dozen lines of the end of the file. so then i tried:

If you read by chunks, you need a while loop to read the whole file step by step. It is meaningless to remove the while loop. Also reading your file alone doesn't crash the app. If you run the code I gave you it doesn't crash the app.