Member Avatar for personx1212

hi there, i have some binary data (zip files, music, etc..)stored in a database, how can i start those files in memory without saving them to HDD..or even to temp files folder...

thanks.

  • 3 Contributors
  • 3 Replies
  • 140 Views
  • 7 Hours Discussion Span
  • Latest Post Latest Post by personx1212

Dani AI

Generated

Quick summary and practical options depending on what you mean by "start" the file.

If you only need to process the binary inside Python (read a ZIP, inspect an MP3 tag, feed an image to PIL), avoid disk entirely and use an in-memory file object. Many standard libraries accept file-like objects — for example, create a bytes buffer and hand it to zipfile, PIL, etc.:

from io import BytesIO
import zipfile

blob = get_blob_from_db()   # bytes from your DB
zf = zipfile.ZipFile(BytesIO(blob))
print(zf.namelist())
content = zf.read(zf.namelist()[0])
zf.close()

For larger blobs that you want to keep in memory until a threshold, use tempfile.SpooledTemporaryFile(max_size=...) so small files stay in RAM and only large ones spill to disk.

If you need to invoke an external program (a media player, external unzip tool) that requires a filesystem path, is right that external processes cannot read your process memory directly. On Linux you can avoid physical disk writes by using a RAM-backed filesystem (as suggested) such as tmpfs (e.g. /dev/shm) or by using newer kernel features (memfd) and exposing the descriptor via /proc/self/fd/N. Those approaches keep data in RAM but require testing: some external apps re-open paths or require seekable files and may not work with every trick.

On Windows there is no single built-in tmpfs equivalent; the usual choices are a third-party RAM-disk driver or writing a temporary file (or using native memory-mapped APIs, but those do not give a regular filename).

Cautions: keep an eye on memory use, test the target application for compatibility (some apps re-open files or check metadata), and always clean up handles or tmpfs entries. Recommended default: use in-memory buffers (BytesIO/SpooledTemporaryFile) when staying inside Python; use RAM-backed tmpfs or memfd/proc tricks only when an external program absolutely requires a filename.

Recommended Answers

All 3 Replies

Member Avatar for jcao219

No, you have to save it to a temp file.

You can't expect software to be able to read your program's memory.

Edited by jcao219 because: n/a
Member Avatar for Gribouillis

If your OS is linux, you should be able to store your file in /dev/shm which is a virtual directory which exists only in shared memory. Example

>>> f = open("/dev/shm/foo.txt", "w")
>>> f.write("hello world\n")
>>> f.close()
>>> f = open("/dev/shm/foo.txt", "r")
>>> print f.read()
hello world

>>> f.close()
>>> import os
>>> os.remove("/dev/shm/foo.txt")
# if we don't remove the file, it will be removed when we shutdown the computer.

Another way to do it is to run a live linux CD and disable swap.

Edited by Gribouillis because: n/a
Member Avatar for personx1212

ok guys, thanks a lot.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.