Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Datastructures for external memory (omega-prime.co.uk)
98 points by nimish on July 8, 2016 | hide | past | favorite | 8 comments


The Intel 750 SSD definitely doesn't get 440k IOPS with a queue depth of 1. It's more like 10k IOPS at that queue depth.

This basic assumption mistake really changes a lot of the numbers in the article.


Why aren't these data structures also good for main memory?

Is the gap between L1/L2 cache vs main memory not large enough? (1ns vs 100ns)


These data structures are good for main memory. I've implemented in-memory B-trees in C#, and they were 2x to 5x faster than .NET's built-in sorted maps. The built-in sorted maps also used 3x as much memory. Surprisingly, B-trees are better even for immutable collections, beating the built-in .NET immutable sorted map by 3x to 5x on updates. This is all with a straightforward implementation.

Red-black trees and AVL trees should not be used any more in my opinion. They are much slower and in my opinion B-trees are easier to understand and implement. It is still customary for data structure courses to teach red-black trees and AVL trees, but they should teach B-trees instead.


The linux scheduler uses red and black trees, there doesn't seem to be any performance problem there. I think that's a pretty ringing endorsement of the dsta structure no?


Did they try a B-tree and found it to be slower?


B-Trees are quite popular in-memory structures when focusing on performance. I expect the others could be adapted as well (and likely have been). A quick google turns up http://www.cs.cmu.edu/~chensm/papers/fpbtree.pdf


This reminded me of BetrFS presented at FAST 15. It looks like there's a more up to date presentation online[1] where the delete performance issues have been fixed.

[1] https://www.youtube.com/watch?v=fBt5NuNsoII


Let RDBMS take care of it




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: