Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support indexing ObjList while connecting to disk #256

Open
kygx-legend opened this issue Feb 24, 2017 · 6 comments
Open

Support indexing ObjList while connecting to disk #256

kygx-legend opened this issue Feb 24, 2017 · 6 comments
Labels
enhancement
Milestone
v0.1.3

Comments

@kygx-legend
Copy link
Member

No description provided.

@kygx-legend
Copy link
Member Author

@lmatz please help this issue for external memory.

@ddmbr
Copy link
Member

ddmbr commented Feb 24, 2017

This issue is a good one!

My idea for ObjList is that it's just an abstraction, allowing users to lookup and traverse. Underlying the interface it could be disk storage or anything. This should make many things much easier.

@lmatz
Copy link
Contributor

lmatz commented Feb 24, 2017

Is it appropriate to make a buffer pool which handles read/write requests from ObjLists and AttrLists, then fetches the pages which are necessary from disk. Sort of like how database does its job. So it may handle the situation where multiple ObjLists or AttrLists are being processed in husky-h4.

@kygx-legend
Copy link
Member Author

We have thought about that but the main problem is how to handle non-pod object in a ObjList. The object can be defined anything via template. The best solution is to transform it into a BinStream. Based on your idea, I think it could work if combining buffer pool with BinStream. It's more complicated to do such an abstraction, which means much time. We want some simple and direct implementation.

@kygx-legend
Copy link
Member Author

@ddmbr yeah, your idea is great and similar with mine. We can start from this.

@kygx-legend kygx-legend added this to the v0.1.3 milestone Mar 6, 2017
lmatz added a commit to lmatz/husky that referenced this issue Mar 7, 2017
@lmatz
[Core] Add an ObjListPool, LRU Cache, FIFO Cache to support swapping …
f6f8c97 
…of ObjLists later

issue husky-team#256
Open a pull request for more discussion and changes.
@lmatz lmatz mentioned this issue Mar 7, 2017
Closed
lmatz added a commit to lmatz/husky that referenced this issue Mar 15, 2017
@lmatz
[Core] Add an ObjListPool, LRU Cache, FIFO Cache to support swapping …
5d969d4 
…of ObjLists later

issue husky-team#256
Open a pull request for more discussion and changes. Temporary version.
lmatz added a commit to lmatz/husky that referenced this issue Mar 20, 2017
@lmatz
[Base][Core][Example][Lib][ML] Add Memory, MemoryPool, Cache, PageSto…
7cbad9f 
…re, Page to support swapping of ObjLists

issue husky-team#256
Modify base/disk_store, base/serialization, core/config, core/objlist, core/objlist_data. Add default constructor to objects.
@kygx-legend
Copy link
Member Author

See https://github.com/kygx-legend/husky/tree/em.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
v0.1.3
Development

No branches or pull requests
3 participants
@ddmbr @kygx-legend @lmatz