I am beginning a master's degree in computer science this coming fall. I have been reading about "big data". Does anyone work or study in this field? I know it is relatively new, but I am curious as to how one would get involved.
coolbeanbob 17 Junior Poster
Rashakil Fol 978 Super Senior Demiposter Team Colleague
"Big data" is when you have more data than typical relational databases are capable of handling efficiently.
It isn't really a field of computer science or something that you can "study". If you're writing applications that use a ton of data, there's essentially the question of what tools to use for storing the data, and the question of what ways in which you can efficiently process the information. The knowledge you need is that of what tools are out there for this purpose, like various "NoSQL" data stores or the practice of "sharding" some SQL databases. Unfortunately some of the "NoSQL" data stores are mediocre software applications written by web developers who thought they could do systems programming.
People only get good at this by doing it, or are already good at it without doing it by the very nature of generally being good at programming.
On the other side there's the question of engineering database management systems for storing these. There's the distributed computing side of this and the storing-stuff-on-disk-and-caching-it-in-RAM side of this. What it takes to be good at that is to not suck at data structures and algorithms and be good at math and be a good systems programmer. This isn't really a new sort of subject, it's been around for a long time. The general tilt of this field depends on the relative price of storage and CPU in terms of component cost and energy cost.
Be a part of the DaniWeb community
We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.