Posts by DGPickett

No RDBMS or ETL app has a magic bullet. Getting data in is the big challenge. Some RDBMS have bulk input for CSV or the like.

One pitfall is that a big insert may trigger long and many row/page/table locks. One middle path is to import in bulk to a temporary table, a staging table, and use row count limits to apply just N rows at a time. Or you can just limit the batch size before any import. Locks may not concern a reporting or batch database, but are critical in a transactional database.

Once I compacted a huge table with clustered indices after a 90% delete this way, but I slept after each mini-batch as long as it took to apply so my DB used was under 50%, and the interactive users never noticed a thing. It took a few days to compact the whole table: copying to a batch table, deleting, reinserting, deleting the batch. The related indices also compacted! I recall setting N to the DRBMS page size in bytes, 2048, to use whole pages in each pass.

On one project, I opened a flat file in a UDF so, with a stored procedure supplying empty rows, I could call related UDFs to fetch fields in an insert-select. In another project, we used java in Oracle to map data like a table in a query for insert-select, which in Oracle can insert to many tables. My one insert-select for 5 tables was 4k long and terrified …

I optimized a Binomial min heap in c++, great for low costs when you discard the heap not empty because you have your spanning tree or shortest route. I changed the root to an array for fast merge.

C is pretty married to ASCII but that leaves 128 more characters 128-255. ISO8859-1 assigns them to support Nordic/Germanic languages, often called LATIN-1. Even in the old days of 6 bit characters and BCDIC, what you printed depended on what roller was in the printer. Later, what belt, even with ASCII! Meanwhile, IBM spurned ASCII, went to EBCDIC, and even they had various code pages to assign glyphs to codes. JAVA went all unicode, and unicode/UTF-8 is ASCII up to 127, then goes on to support most popular languages and uses more characters, variable multiple characters in UTF-8. Even with unicode, the glyph you get these days depends on the font. Google up some wiki pages.

Generally any column named id is an artificial, auto-incremented, indexed key. Are your id's also primary keys, and so indexed? If a btree index, maybe make a hash index?

A order date index btree not hash allows a range selection.

Assuming the file is utterly consistent (3 line sets, ssn w/o dashes in the second line), you can toss a line, read a line as a string or int, toss a line in a 3 line loop. Now, you can drop you SSN into an (ordered) set, which eliminates duplicates and sorts, either as the string or an int (also fits in a long, unsigned int, unsigned long -- 9 digits is 3 commas is 30 bits). Then dump the set to your output.

If there are file variations or intermittent dashes, they you need to sense the good lines and chase out the dashes.

I write all my code in my brain as pseudo code, and later encode it in the language of the moment. Sometime I use paper in the middle.

I have a bluetooth keyboard or two that make my phone pretty useful for data entry and programming. It is nice to have arrow keys and similar keyboard driven shortcuts.

There are online compiler sites (some buggy). They allow you to test some stuff without installing a compiler/language.

Assuming there is no magic free direct conversion tool, convert the python to pseudo code and then to JAVA. This will test your knowledge of when python does nice things for you automatically, which you may need to do manually in JAVA. You may be able to use more appropriate containers in this process.

It depends on locks already held. Deadlock is p1 gets a and wants b bu about the same time p2 gets b and wants a. If all threads always lock resources in the same order, there is no deadlock or starvation.

I usually go with another int set to -max (0x80000000) when seeking a max, so the first is generally greater and replaces it. Using <= means equal values get copied, and as writes are more expensive than reads (think cache), why bother unless either you are required to get the last not the first, or -max is actually in the array?

You need to return the index after the loop, the major bug that the error is indirectly warning you about. More generally, even if you know some return in a loop or conditional block will fire always, you will get this warning unless there is a return before the ending curly brace, so either say 'break' or let the loop run out, and let the code fall on it.

The newer 'find' has a batch execution mode, but I find it easier to use 'xargs'. I usually do 'xargs -n101 ...' to both prevent execution with no arguments and get the work started while still getting 99+% of the bulk savings. In fact, I wrote a fast xargs that does the work of one batch while collecting the input for the next batch without excessive blocking. For the power user, xargs has a high steroids cousing, parallel. Or you can spin off the rm in the bg since there is no output as 'rm some_file &' or batches 'rm some_files &'. Sometime I use:

find ... -type f -name '*.css' | xargs -n101 echo rm | while read l
    do
        $l &
    done

but you have to beware of spaces and such in the file names!

You want to 'find ... -type f', so you are not working on directories, devices, or sym links! When not trying for directories, the '-r' option of rm is not needed.

Once you have the average, you need another loop that reviews the input against it and prints the desired items.

No comma before where in SQL.

Or better yet, make it an array of char*, since a double quoted literal value is a char pointer to a constant on the heap.

Not seeing a lot of morphine, just a loop, a few ints, some io!

identical switch within switch is just wrong, and printing inside swtch ouside any case is not at all nice.

More values than columns!

Generally, when you go to a web site without a valid timed login cookie, the server redirects you to the login page, possibly with a cookie or variable saying where to go (the original page) after login.

You need to track both the number of cells used (top for a stack) and the capacity (max cells allocated) so you know when to expand the container (stack). Evict your methods from the class, removing class:: and call it a struct.

Since auto resizing arrays are not there in c, not classes, you might want to use a linked list or realloc() a char[] to size.

If you open an output ofstream and use it not cout, the data is saved in a file.

If stick 3 runs the machine OK alone, then incompatibility is a strong possibility. However, there is usually one way things can work and an infinite number of ways they can be broken! It might be broken so it runs but messes up any and all other cards on the same bus. I recommend buying a completely identically spec'd set of ram sticks of the highest performance the cpu and motherboard support. The pairs can be of different sizes, but within the pairs they should match so they can be interlaced/channeled. Go for the max size your system supports. Parity and error correction, if supported, are nice but not required for most of us. Clean the fans, too! :D

Somehow, the last time i added an equals() to a JAVA List (not Set, as Set does not support lookup like List and Map), it got ignored. I wrote my own findXXX(). Your results may vary! I'd recheck, but it was in a student's work. I should write something to try again! Still not sure why Set does not want to support lookup. I suppose it meets some math ideal!

EOF check between read and write, use get() to pick up white space characters, getline() removes the lew line.

dgp@dgp-p6803w:~
$ myCC c++rw
dgp@dgp-p6803w:~
$ c++rw
Type in file name followed by the extention ".txt"
x.txt
File created.
Please enter text: 
hohoho
File Information: hohoho
dgp@dgp-p6803w:~
$ 

#include <iostream>
#include <fstream>
#include <sstream>
#include <string> 
using namespace std;
int main()
{
    cout << "Type in file name followed by the extention \".txt\"" << endl;
    string user_file;
    getline( cin, user_file );
    fstream file; //object of fstream class
   //opening file "sample.txt" in out(write) mode
    file.open(user_file, ios::out);
    if (!file)
    {
        cout << "Could not open file" << endl;
        return 0;
    }
    cout << "File created." << endl;
    //write text into file
    string user_input;
    cout << "Please enter text: \n";
    getline(cin, user_input);
    file << user_input << endl ;
    //cout << "For testing. User input is: " + user_input<< endl;
    file.close();//close the file
    //system("pause");
    //open file to be read
    file.open(user_file, ios::in);
    if (!file)
    {
        cout << "Could not open file" << endl;
        return 0;
    }
    char ch; //read character
    cout << "File Information: ";
    while ( true )
    {
        file.get( ch );
    if (file.eof())break;
        cout << ch;
    }
    file.close(); 
    return 0;
}

Lots of references on the dangers and costs of various equalities. This one seems very nice: https://www.mattzeunert.com/2016/01/28/javascript-deep-equal.html

Just like scanf() and the JAVA Scanner Class, when you >> the next string, int, double, char, etc., then it reads, initially skipping over white space, into the variable from input until it has what you want, and when it hits an incompatible character, like white space for a string, it pushes that character back into the IO buffer. http://www.cplusplus.com/reference/istream/istream/operator%3E%3E/

So, your line feed gets pushed back and waits for the next cin call. This is way common, but you can just read a char or do getline() to flush out the linefeed, or do a cin.getline() for the file name, which discards the linefeed. http://www.cplusplus.com/reference/istream/istream/getline/

I have had good luck with:

"<HTML><HEAD><TITLE>My Title</TITLE>...</HEAD><BODY>...</BODY></HTML>"

Wow, I just write a service that runs the query and reformats the column titles and values into an HTML table. If I wanted it to be more spreadsheet-like, there are caned javascript bits out there to allow you to re-sort the table.

In a database, you can just keep the count of every part, and as the users take or return stock, the count changes. If you want to know about the activity, you can have a table of activities, which can either be the primary and the stock level is the sum of counts for any product from original inventory forward, or a trigger or acess procedure generated side table of history of changes to the stock table. More sophisticated systems tie invoices to stock changes, so you can return a whole order at once and similar nice additional behaviors.

The easiest thing is to write CSV files, which follow 4 simple rules: 1) cells are separated by a comma, 2) rows are separated by a cr-lf, 3) unless the comma or cr-lf is enclosed in double quotes, and 4) a doubled double quote is a double quote in a cell, not an enclosing double quote.

There are libraries that write excel files .xls or .xlsx, visual whatever, perl, c, JAVA. I would pipe to a supporting app if it was not linkable. One would think the xlsx, which is XML, would be easy to write, but MS!

Wow, can of 'sorts'. You need some sort of container, and there are others besides array: linked list, tree, hash table, skip list. Tree sorts as it adds, and even C has tsearch(). For small sets, you can delete the min value when you find and print it, and scan all for the min repeatedly. Fancier sorts often divide and conquor, as an array of N random numbers, is an array of N sorted lists of 1 item each, which can be merged into N/2 lists of 2 sorted items, and so on.

== is a shallow comparison, like shallow copy, true only if the refrences are identical, not the objects. http://adripofjavascript.com/blog/drips/object-equality-in-javascript.html JAVA and C++ also have this sort of limitation unless you overload the == operator or use a method, like compareTo, equals, and JAVAScript isEqual. JAVA Sets use object.equals(), which is furstrating as you cannot override with your own class.equals().

There is consulting, where they run interference for you, freelance work, volunteer work, teaching, tutoring and open source collaboration to get your status up and your skills sharp.

Google sheets keeps track of all changes and who entered them, and allows multiple user concurrent access. An RDBMS app needs to know who is logged in to make changes, and could have a complete history as well as status (the latest history) rows and active (the status is active) rows. I once wrote a quick app like this, where active and status were just views of history, all churn ws insert, and so history was 100%, very handy in RAD. Later we added archival update/delete, where the original/deleted were copied to parallel tables, and undo/redo was supported. It had just a few tables: courses, students, teachers, and learning events where the three interacted. Later we added areas, which was nice until users deleted people to get them out of their area only to have them reappear when they were reactivated in their new area. Dead to me is not dead! :D

Hash tables are containers. Hashing creates an integer from other types, usually the characters of a key, where it represents the values and positions of those characters. One divides the hash by the bucket count to select a bucket, which is a simple container like linked list where you can search for a precise match. One item per bucket is unwise! The cost of a bucket is a pointer, or 4/8 bytes, so use lots of buckets. Mod-2 bucket counts can be mod by simple and-masking out the lower bits, and the bucket count can be doubled (linear hash table) but the misplaced copied items must be culled or tolerated. Non-mod-2 bucket counts may randomize better! Hash is faster than tree, hash generate/divide cost being less than log2 comparisons, but it is not sorted. Other containers are the skip list: a tree-linked-list hybrid, the trie (a tree with one level per key character), the array and the linked list. Each has its advantages. The linked list adds/expands cheaper. Array lists always know their size, but are expensive to add or delete intermediate values. Linked list search/delete/insert requires comparing half the items on average. Lists are often used as queues in multithreaded situations, and one needs to decide on a fixed size or expensive expansion times, anticipate locking costs, thread suspension costs.

The join table to itself tactic is very good for converting rows to columns. There are other tricks you can use with aggregates and case statements.

The major methods for image reduction in size are compression, like in jpeg, and resampling. Resampling may take something like a weighted average, which can make things muddy, or select the pixels that are closest to the original position, a center-of-pixel strategy where a minor artifact may grow in relative size. Some strategies do some sort of avaraging that is center-biased. It is nice to keep the original around and generate all rsized images off it, as repeated resampling adds to the muddiness.

DHCP operates a pool of IPs for clients that need them. Usage is a bit worthless unless they stay assigned a long time. As noted, the mac from the ethernet layer is a better id.

The Securities/Stock trading crowd like c++ for the speed, and XWindows was there long before most others.

How do you teach those who fall off the official wagon?

The hollow diamond is about distance from center column and distance from center row; their sum s/b half the height to print. In this case, 3. So:

for ( int c = 0 ; c < 7 ; c++ ){
  for ( int r = 0 ; r < 7 ; r ++ ){
     If ( r + c == 3  || r - c == -3 || r + c ==  9 || r - c == 3 ){
        cout << ‘*’ ;
      }
    }

     cout << ‘\n’ ;
   }
// r + c goes from upper right to lower left
// r - c  goes from upper left to lower right

Maybe put the array and indexes in parens after the & ? Debug messages on stderr , which, unlike stdout, is unbuffered!

There are just 4 rules to parsing a CSV:
Commas separate fields
Carriage return linefeed separates rows
Unless inside “
But “” is a literal “

A palindrome check should be a loop through the first half of the string or 1/2 less checking the mirror byte for a match. To make it a number check, filter for numbers along the way or format number types to a string. For instance, for 9 places check 1 with 9, 2 with 8, 3 with 7, 4 with 6 and who cares about 5! For 1 through length>>1 char n == char length -n +1, with the usual care about the cardinal ordinal numbering of char in a string or array.

JDBC tools like SQuirreL and jisql let you extract schema and data. I like to use LINUX sed to pipe a select into an insert, no intermediate files.

To read a file, you need to declare an ifstream, open it, read from it using using >> into the appropriate types and check for eof(), bad(), fail() to detect errors and end of file. Note that >> at EOF sets fail (you didn't get what you asked for) as well as eof but not bad.

Putting the data into parallel arrays is pretty easy. Declare an array for each of the 3 items, and load it as you read using the same index. The arrays need to be big enough or you need to dynamically size them, google for examples.

Comparing null terminated char arrays is:

#include <string.h>
int strcmp(const char *s1, const char *s2);

but for known length unterminated strings:

   #include <string.h>
   int memcmp(const void *s1, const void *s2, size_t n);

Both return an integer less than, equal to, or greater than zero if s1 is found, respectively, to be less than, to match, or be greater than the first n bytes of s2. Beware: other comparisons do other things, useful in other cases! For instance, for case insensitive null term string comparison:

   #include <strings.h>

   int strcasecmp(const char *s1, const char *s2);

The keyboard and monitor data is 'cooked', not raw, unless you turn on raw orocessing like with stty in UNIX. The enter key is a carriage return 0x0D or ctrl-M, but is cooked into carriage return and linefeed 0x0D0A ctrl-M ctrl-J. On the teletype, one makes the typehead go to the left end (slower, so a head start), the other makes the paper roller move (still slow). A monitor emulates a tty printer. The real tty needed padding characters to allow time for the motions, too! See 'man stty'!

There may be integration to get the drivers and decoders onto the same die as the LEDs, but yes, otherwise, 3020 connections x 3 (R G B) and a very fast scan rate. Any my TV is 4K, not 1080. I suspect they may be driving more than one dot at a time, as 16 (for 4k is 16 x as many pixels) x 1080 x 1940 x 59.98 Hz = 2 GHz, but maybe -- chips are so fast these days. All three colors need to be driven at once, I expect. Pixel intensity can be modulated by voltage or time of the pulse, and varying the timing or voltage at high scan rates is hard.

For masking, write the random numbers directly, not as ascii. Write with fwrite() so each number is buffered into large block writes. Note that RAND_MAX is just 31 bits:

/usr/include/stdlib.h:#define RAND_MAX 2147483647

You can write 3 bytes at a time to have pure randomness. Remember that the bytes you want are located at different ends on little-endian hosts like x86 and big-endian hosts like SPARC. The middle 2 bytes are safe!

I would think that, for masking old data, a random block of a handy size like 65536 would suffice, written over and over to make a large file. More masking on magnetic media takes more write passes with different random data.

Many write passes might shorten SSD life, but I do not think SSD data are possible to recover after a rewrite.

Depending on the file system, you may be limited to 4GB files, but just open a new file and keep on. Systems may get unstable if there is no space left on the primary disk partition, so automated deletes immediately at write error would be nice.