Great article, have you also tried https://github.com/vaexio/vaex an out of core parallel dataframe library for Python? It can process around 1 billion rows/second (disclaimer: I’m the main author)