Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

What do you mean by SQL?

In my experience, once you're hitting hundreds of millions of records, implementation details of your database engine will start to matter. A database designed for transactional workloads like Postgres will start to choke on aggregate and window functions, often taking minutes to run instead of milliseconds. A columnar database like Redshift (which exposes a SQL interface) will breeze through it without a sweat.



Of course - I was comparing SQL to loading all the data in memory and then using one's programming language of choice to do the hard work. Even a poor schema in a row-based database will outperform what I described after a certain number of records.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: