A Bespectacled Engineer's IT Journal

I Built an Ultra-Fast CSV Data Retrieval Library

2026-08-21

It's called getdatafromcsvbuffer — a library that retrieves data from CSV files without ever touching V8's Heap memory.

Why Are Existing CSV Parsers Slow?

Most existing CSV parsers use V8's Heap memory — for example, to convert raw bytes into strings. Objects managed by V8 carry metadata, pointer information, and other overhead as part of their data structures.

To work with CSV data as arrays or objects, you need to load everything into the V8 Heap. That takes significant time and memory.

The Approach: Operate Directly on the Buffer

Node.js is built on C++, and the Buffer area exists outside of V8's memory space — it's a C++ memory region. Node.js exposes interfaces like buffer.indexOf to manipulate this Buffer area directly.

When you read a CSV file via FileStream, the raw byte sequence is stored in a Buffer. Instead of converting it to a string and parsing it, getDataPointRowHeaderName operates on that raw Buffer to extract only the data you need.

Usage

getDataPointRowHeaderName(buffer, row, headerName)

How Fast Is It?

I loaded the following CSV and retrieved the country value from row 5, comparing against csv-parse.

name, age, gender, country, city, company
Lily, 24, femal, USA, NewYork, MS
Daniel, 27, male, Dutch, AS, META
John, 29, male, USA, NewYork, MS
POLE, 56, male, Brazil, Sanpauro, META
emily, 45, femal, France, Pari, AWS

Environment: Windows 11 / Node.js v26.5.0

getDataPointRowHeaderName csv-parse
Run 1 2.625ms 4.399ms
Run 2 2.81ms 4.368ms
Run 3 2.509ms 4.288ms

Even on a small CSV with a single lookup, it's over 1.5x faster. The gains should be even more pronounced with larger files and heavier workloads.

Available on npm and GitHub

Feel free to check it out — even small comments or issues are very welcome!

← Back to list