Skip to content Skip to sidebar Skip to footer
Showing posts with the label Bigdata
Python - Parsing A Text Onto Columns By The Position Of Each Item

Python - Parsing A Text Onto Columns By The Position Of Each Item

Post a Comment
The Bovespa (brazilian stock exchange) offer a file with all the quotes in a timeframe. The file is… Read more Python - Parsing A Text Onto Columns By The Position Of Each Item
Incremental Pca On Big Data

Incremental Pca On Big Data

Post a Comment
I just tried using the IncrementalPCA from sklearn.decomposition, but it threw a MemoryError just l… Read more Incremental Pca On Big Data
Correct Way Of Writing Two Floats Into A Regular Txt

Correct Way Of Writing Two Floats Into A Regular Txt

Post a Comment
I am running a big job, in cluster mode. However, I am only interested in two floats numbers, which… Read more Correct Way Of Writing Two Floats Into A Regular Txt
Get A List Of Subdirectories

Get A List Of Subdirectories

Post a Comment
I know I can do this: data = sc.textFile('/hadoop_foo/a') data.count() 240 data = sc.textFi… Read more Get A List Of Subdirectories
How Can A Reduce A Key Value Pair To Key And List Of Values?

How Can A Reduce A Key Value Pair To Key And List Of Values?

Post a Comment
Let us Assume, I have a key value pair in Spark, such as the following. [ (Key1, Value1), (Key1, Va… Read more How Can A Reduce A Key Value Pair To Key And List Of Values?
Numpy: 3-byte, 6-byte Types (aka Uint24, Uint48)

Numpy: 3-byte, 6-byte Types (aka Uint24, Uint48)

Post a Comment
NumPy seems to lack built-in support for 3-byte and 6-byte types, aka uint24 and uint48. I have a l… Read more Numpy: 3-byte, 6-byte Types (aka Uint24, Uint48)