scipy - Python multi dimensional sparse array -

i working on project need deal 3 dimensional large array. using numpy 3d array of entries going zero, it's lots of wastage of memory. scipy sparse seems allow 2d matrix. there other way can store 3d sparse array?

scipy.sparse has number of formats, though couple have efficient set of numeric operations. unfortunately, harder ones extend.

dok uses tuple of indices dictionary keys. easy generalize 2d 3d or more. coo has row, col, data attribute arrays. conceptually then, adding third depth(?) easy. lil require lists within lists, messy.

but csr , csc store array in indices, indptr , data arrays. format worked out years ago mathematicians working linear algebra problems, along efficient math operations (esp matrix multiplication). (the relevant paper cited in source code).

so representing 3d sparse arrays not problem, implementing efficient vector operations require fundamental mathematical research.

do need 3d layout vector operations? you, example, reshape 2 of dimensions 1, @ least temporarily?

element element operations (*,+,-) work data of flattened array 2 or 3d version. np.tensordot handles nd matrix multiplication reshaping inputs 2d arrays, , applying np.dot. when np.einsum used on 3d arrays, product summation on 1 pair of dimensions (e.g. 'ijk,jl->ikl')

3d representation can conceptually convenient, can't think of mathematical operation requires (instead of 2 or 1d).

overall think you'll more speed reshaping arrays trying find/implement genuine 3d sparse operations.