Member Avatar for gunbuster363

When I am processing the data, I put them inside some beautiful data structure for convenience, but then again I am done with my work, and I want to save the result in hard drive, I need to store them in a txt file, and for later usage, I need to write them in a structured way(in text form).
And then again I need to use the data processed, which was store in a txt file, I need to read it again and I need to write code to store them in some data structure.

Isn't this non sense?

  • 4 Contributors
  • 5 Replies
  • 117 Views
  • 7 Hours Discussion Span
  • Latest Post Latest Post by sahiti

Dani AI

Generated

This is a very common workflow: nice in-memory structures, then saving them for later use. It is not nonsense — it is solved by choosing a persistence/serialization approach that matches your needs. pointed out that you can persist whole Python structures; below are practical trade-offs, safer choices, and small examples so you do not have to reimplement parsing every time.

If your data is just nested dicts/lists/strings/numbers, prefer a text, portable format (human-readable and cross-language). JSON is the usual choice in that case — human-friendly, widely supported, and easy to inspect or edit. Example pattern in Python:

import json

payload = {"version": 1, "items": my_dict}
with open("data.json", "w", encoding="utf-8") as f:
    json.dump(payload, f, ensure_ascii=False, indent=2)

with open("data.json", "r", encoding="utf-8") as f:
    payload = json.load(f)

See the Python json docs for details: json — JSON encoder and decoder.

If you need to preserve arbitrary Python objects or full object graphs, Python’s binary serializers will do that, but they come with two key caveats: they are not safe to load from untrusted sources, and compatibility can break across Python versions. For a simple on-disk dict-like API use shelve; for relational queries use SQLite; for large numeric data use HDF5/numpy formats. See: pickle — Python object serialization, shelve — Python object persistence, sqlite3 — DB-API for SQLite.

Practical tips: include a small version or schema field in saved files and write migration code for future formats; write atomically (save to temp file then rename) to avoid corruption; compress if size matters; never unpickle data from strangers. For : pick JSON for readability/portability, a DB for queries, or a binary serializer for fidelity — each avoids the repeated hand-parsing you complained about.

Recommended Answers

All 5 Replies

Member Avatar for vegaseat

You save your entire data with module pickle. Here is an example ...

# use module pickle to save/dump and load a dictionary object
# or just about any other intact object
# use binary file modes "wb" and "rb" to make it work with
# Python2 and Python3

import pickle

# create the test dictionary
before_d = {}
before_d[1]="Name 1"
before_d[2]="Name 2"
before_d[3]="Name 3"

# pickle dump the dictionary
fout = open("dict1.dat", "wb")
# default protocol is zero
# -1 gives highest prototcol and smallest data file size
pickle.dump(before_d, fout, protocol=0)
fout.close()

# pickle load the dictionary
fin = open("dict1.dat", "rb")
after_d = pickle.load(fin)
fin.close()

print(before_d)  # {1: 'Name 1', 2: 'Name 2', 3: 'Name 3'}
print(after_d)   # {1: 'Name 1', 2: 'Name 2', 3: 'Name 3'}
Member Avatar for gunbuster363

it seems great! I will try it later~ thanks!

Member Avatar for sahiti

hi diz sahiti, i actually didnt understand wat u wrote,im new 2 dis forum, hey can u tel m abt get( ) function in C............

Member Avatar for hondros

@ Sahiti:
Please learn netiquette and some english, not "eubonics", or "txtspeak" on online forums. While we will help you, we would like if you spoke as you would in a normal conversation. Also, if you need help with the 'C' syntax/language, please ask in the appropriate forum.
Thank you :D

commented: nice way to handle this +10
Member Avatar for sahiti

im sorry ,i will never repeat this again..............:$

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.