Member Avatar for zahmatkesh

hi
i have a table with 1 milion record.
i want to sort and read in 1000 groups.
i can read diffrent groups but i want to apped 2 datset to each other.
how can i do that?
pleae help me.
thanks

  • 4 Contributors
  • 4 Replies
  • 114 Views
  • 1 Week Discussion Span
  • Latest Post Latest Post by BitFarmer

Dani AI

Generated

: first confirm where the million rows live (database table vs file/in-memory structure) — as asked, that changes the answer. Also note what you meant by "append 2 dataset to each other": do you want one combined result for display, or to persist the merged rows back into a table? and were right to push server-side solutions when a DB is involved; doing sorting and paging in SQL avoids loading the whole million rows into memory.

If the data is in a relational DB, fetch 1,000 rows at a time using a stable indexed sort column and keyset pagination (OFFSET on very large offsets is slow). Example (MySQL/Postgres style):

SELECT * FROM MyTable
WHERE (SortCol > :lastCol)
   OR (SortCol = :lastCol AND id > :lastId)
ORDER BY SortCol, id
LIMIT 1000;

Repeat, setting :lastCol/:lastId to the last row from the previous page. This keeps each page fast and ordered.

If you must append two TDataSet-style datasets in Pascal/Delphi on the client, copy rows programmatically into a destination dataset with matching fields. Disable UI updates while copying and use a transaction if you write back to the DB. Example routine:

procedure AppendData(Source, Dest: TDataSet);
var
  i: Integer;
begin
  Source.First;
  Dest.DisableControls;
  try
    while not Source.Eof do
    begin
      Dest.Append;
      for i := 0 to Source.FieldCount - 1 do
        Dest.FieldByName(Source.Fields[i].FieldName).Value := Source.Fields[i].Value;
      Dest.Post;
      Source.Next;
    end;
  finally
    Dest.EnableControls;
  end;
end;

Cautions and tips: ensure identical field definitions (types and names), avoid primary-key conflicts when inserting back, commit in batches for large writes, and prefer server-side merging (INSERT..SELECT or provider-level bulk methods) for performance. If using a client dataset component, check its built-in bulk-load or provider methods to avoid row-by-row work.

Recommended Answers

All 4 Replies

Member Avatar for cao

You did not state what type of db table you are using but have a look at the sql command union

Member Avatar for FlamingClaw

one million records....that's nice .....
where do you store them,in a file or in the memory(is this a linked list?)?

Member Avatar for BitFarmer

If you have ONE table, you can't append two dataset... something is not right in your question... if you have " tables and want to mix them into a single query, you should try UNION, as far as you are accesing those record via SQL.

With the info you sent, that's all I can say!

Member Avatar for BitFarmer

If you have ONE table, you can't append two dataset... something is not right in your question... if you have 2 tables and want to mix them into a single query, you should try UNION, as far as you are accesing those record via SQL.

With the info you sent, that's all I can say!

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.