-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Set up website + document updates automatically #3
Comments
Any chance I could join in if/when you and @vsoch meet about this, @rhine3? I would like to set up something similar for https://github.com/vocalpy/pollyglot and it might help to hear you both talk it out. |
That would be ok with me! I pinged @rhine3 via email end of June but no response yet, so likely busy. If you want @NickleDave we can also just meet to spec out what you have in mind first - whatever the order turns out to be one can definitely help Toni fluency the other! |
👍 thank you @vsoch -- I think @rhine3 might have been hanging out with some birdies this last week and will probably be back on-line presently. My project is definitely low on the to-do list, buried underneath a ton of neglected maintenance work on other projects. |
Issue opened! rseng/rse#59 It's on my TODO list so hopefully will start this weekend or an evening next week. Thanks y'all for meeting it was fun, and keep me in the loop! |
hey folks! I started the export UI this weekend! Based on our discussion I'm assuming the main design should have a searchable table - but do y'all have other preferences? Point me to sites you like the design of? I'll try to make it customizable (via the jekyll template) but ideally I'll start with a nice layout / theme that most people like. |
okay looking good so far! I have added an export functionality (no PR yet, still tweaking) and started a template with a nice home page for a description / logo, and very basic software page with a table (akin to the rseng software but with annotation fields removed). I've also done search... and a very basic json API to get metadata for all the software in the database: I think the average site is going to maybe want news (e.g., for posting news or blog articles) and then an about page - and a quick question for y'all - do you want each collection entry to have its own page, and if yes, what would you like it to look like? It could literally be another table with more metadata (or something else)... I wanted to ask first before doing anything. Also I found this kind of cool editor template that we could use in some cool way with the local cache - my dummy example was to install things into a custom made Dockerfile, but you could imagine just navigating around the site and making a custom list of software to export? Do something else with? Let me know your thoughts! Still lots to do, but I promised I'd keep you updated! Will post again when I have more (and of course let me know your thoughts to the above!) |
Update - vision for future! When we have more organized tags, let's add a visual like this -> rseng/rse#60 |
ooh pretty |
Cool visualization for tags - great work so far, thank you Vanessa! I think a simple clean table looks good. I like the visual style of Elicit (which is also a cool tool worth checking out). |
@louisfh do you know what table framework they use under the hood? I can't see an actual table without an account I don't think. Basically if you view the source and look at the javascript and javascript libraries - does a table framework jump out? |
Some more tiny progress! I made a dummy spreadsheet tonight and have the basic import done: The reason there are only 8 is because I'm requiring fields for a Url, Title, and Description, and the majority in the sheet don't have a formal title. I might just allow that? Haven't decided yet. I do think having some minimal information for a person to parse is pretty important (e.g., a title and description with a URL seems like a good set!) I just need to finalize the import structure for a "custom" parser entry (meaning it's not from GitHub or GitLab) and then I should be able to add this feature, and continue on to the next! At the end I'll be able to make a custom database from your spreadsheet, and then have it update a site nightly along with re-import (which I'll write workflows for). |
Okay import from Google sheet is done: |
Alrighty! So I think I've added the major features we discussed? https://twitter.com/vsoch/status/1548443989539377153 I will still look into the extra for run citation parsing, but for the rest (addition of a schema / update of the analyzer) I think I'll probably need y'alls help since I know nothing about this space! For easy access here is the issue we were following: rseng/rse#59. And here is a complete thread with updates: https://twitter.com/vsoch/status/1548443989539377153. I wound up making a demo sheet that others could copy as an example to import from csv. And nooo rush!! I wanted to get this ready for you asap, and I'm here when you need me for the rest :) @NickleDave there should be enough docs in the example site if you wanted to give it a go for your own project, and of course give feedback or ping me for any wonky JS/css/html help. I will update again if I find out anything interesting for parsing Google Scholar / references (never tried it before!) |
Excellent thank you @vsoch! -- will do |
Hi all, I have a branch in my fork where I was trying to automate generating the database + serving a site: @vsoch I was able to get Also re: serving the site: Thanks! |
@NickleDave you don't need rse start, that's for the flask server. You just need to generate the database, and then copy this repository to grab and update from it. https://github.com/rseng/web |
Or just copy this entirely :) https://github.com/rseng/rse-jekyll-web/ That repository README has full instructions. |
🙌 Thank you! Is there a link to that in the |
Yeah that would be great! So the specific import/export are documented on the commands page https://rseng.github.io And yes, would love a PR! |
👍
I did find the command although finding "commands" itself was not obvious to me. I mainly meant adding a link to the repository you shared.
👍 |
Reporting back here that I was able to generate a static site by just doing I also tried generating a full database for the site using a copy of the entire spreadsheet but hit a bit of a bug there. |
Tweaks to add links to tutorial, and bug fix rseng/rse#75 |
Hi all, just wanted to update you where I'm at with this: Thanks to a ton of help from @vsoch I am at least able to build a site: Tada, 🎉 This is the copy of @rhine3's sheet that I am using to be able to do this The main issue right now is that a bunch of the links in the 'url' column are "malformed" according to the @vsoch you have done so much work already that I am afraid to suggest this, but I wonder if it would make sense to add an importer/scraper that can take a .csv directly? This way the data could be munged + cleaned, e.g. by a script, before handing it off to I have a vague idea how this might be possible in |
yep happy to do that. |
Fantastic work on this @NickleDave!! Thanks so much to both you and @vsoch for making this project happen. Sounds like the finish line is in sight 👀 |
All set! rseng/rse#81 |
Included in release https://pypi.org/project/rse/0.0.46/. |
Hi @rhine3 I went ahead and made a PR for this. @vsoch very patiently worked through a couple minor hitches with me but we have got it going now. If you were to merge now it would add the site, but generated from a copy of the spreadsheet I made, and still with the original RSEncyclopedia template (:thinking: forgot about that part, we probably need to address that). I think we could just get it to generate directly from your spreadsheet if you were open to doing the following:
If you're willing to make those changes, then we can change the url in the GitHub actions file to point directly at your spreadsheet and not have to worry about cleaning up the csv manually. After the fixes @vsoch made we were able to parse all 70 entries. Let me know how to proceed. Happy to close the PR and make a new one later if we need to backtrack a bit. |
Actually I'm lying. We just need to change the logo and text in the template as the very clear instructions that I did not read tell us to. https://nickledave.github.io/bioacoustics-software/
|
David, I made those updates to the spreadsheet and created a website-dev branch, so would accept a PR into that branch from your addRSE branch whenever you're ready! Thank you so much! From there I'll work on the following:
|
Yay 🎉
Done! |
Hi @NickleDave and @vsoch - I was able to make those two updates listed above. There are definitely a few things left to update, but I'm trying to give this repository a DOI ASAP as I am planning to reference it in a paper! So here's my remaining to-do list for accountability:
|
Made obsolete by #13 |
As discussed in #1 - it would be great to create a filterable website (and possibly an automatically updated README?) for the bioacoustics software list. @NickleDave suggested potentially basing this off of the system that @vsoch used to create the Research Software Encyclopedia.
The text was updated successfully, but these errors were encountered: