GitLab

@klaus, thanks for posting. Continuing our discussion on jabber here is first round of feedback on API - to how this functionality is presented to user.

bigarray.array_ram module is ok (I would prefer for everything to live in just bigarray, but python does not allow that easily);
we rename RamArray -> RAMArray (RAM is abbrev.);
for RAMArray constructor: we drop blksize=argument - it is not needed/used for RAMArray (we want RAMArray and ZBigArray to provide the same interface but creating different interface providers can be different);
we don't guarantee for now whether RAMArray will inherit from BigArray or not (I need to sort this out, but likely you don't need to depend on this).

with the above changes it is ok to start to depend on bigarray.RAMArray and use it practically in your applications.

About the implemetation - things like:

+Other than BigArray.resize(newshape), RamArray.resize() copies data internally +so a previously created view will not see changes made to the data after +the resize.

 if ArrayClass == BigArray:
     assert a[0,0] == 2
 elif ArrayClass == DummyRamArray:
     assert a[0,0] == 1

show that currently RAMArray and ZBigArray don't have the same semantic, and by default I would say that should not be like so - i.e. if we start with the goal of providing something that is in-RAM only but have the same semantic - it should have the same semantic fully.

I need to think a bit about whether we indeed need it, but offhand it indeed looks like we better change the implementation to mmap something from /dev/shm from where we can create several mappings that will be coherent to each other.

Please let me know your thoughs,
Kirill

Added 1 commit:

e0899d6b - Add RamArray

@kirr, thanks for feedback.

we rename RamArray -> RAMArray (RAM is abbrev.); done

for RAMArray constructor: we drop blksize=argument done

I agree that we want to have same semantic in RAMArray and ZBigArray.

@kirr and @klaus, let us push this MR.

We have way too much code here and there and this increase cost of project maintenance

@Tyagov we cannot merge right now because 1) RAMArray does not have the same seamantic for data sharing and 2) the patch messes up BigArray tests.

I will try to redo RAMArray based on mmap(/dev/shm/...) and come back with that.

Thank you @kirr .

mentioned in commit kirr/wendelin.core@c4353cc7

mentioned in merge request !9 (merged)

So I've came up with !9 (merged). @klaus, please let me know whether that works for you. Thanks.

(appologize for keeping you waiting for so long)

@kirr great, !9 (merged) is exactly how it should be! It is running now on production without problems. For me, we can close this and merge !9 (merged) /c @Tyagov

closed

@klaus, thanks for feedback. I will have another look at patch at !9 (merged) with a fresh eye again tomorrow, and if ok (hopefully) merge it to master. Please speak up if there are problems or whatever with it.

mentioned in commit kirr/wendelin.core@fc9b69d8

mentioned in commit 99b91c84

Add RamArray