Failing to getfield(Main, ...)

[joa-quim]

I have several of these errors when building GMTjl_doc via GH actions. I do not get them when running the same commands locally (both Windows & Linux)

evaluating code [example_figure] in (examples\plotting_functions\05_1_stats   
+ Franklin Warning: in <examples\plotting_functions\05_1_stats.md>
¦ There was an error of type 'UndefVarError' when running a code block.
¦ Checking the output files 'D:\a\GMTjl_doc\GMTjl_doc\__site\assets\examples\plotting_functions\05_1_stats\code\output\example_figure.(out|res)'
¦ might be helpful to understand and solve the issue.
¦ 
¦ Relevant pointers:
¦ - evaluation of Julia code blocks: https://franklinjl.org/code/
¦ 
¦ Details:
¦ UndefVarError: scatter! not defined
¦ Stacktrace:
¦  [1] add_opt_module(d::Dict{Symbol, Any})
¦    @ GMT C:\Users\runneradmin\.julia\packages\GMT\M9Rtt\src\common_options.jl:2557

The error seems to come from this line

fn = getfield(Main, Symbol(string(this_symb, "!")))

where it errors saying that functions that do exist and and exported are not defined

[tlienart]

Hmm yes that's going to be hard to debug/fix. Do the scatter (without !) work? The getfield(Main...) is probably not ideal because it makes assumptions about what Main is but that's not going to help you much right now.

In the meantime, a relatively easy workaround would be to generate the website locally since that seems to work fine, call optimize etc so the prepath is specified correctly, and then push the content of __site to gh-pages. It adds a manual step but at least will get your site up and going. You could use the publish function to do this, or maybe just look at the code in publish to have your own script that does it if you prefer.

Note: I'm aware the way code is executed in Franklin is not super robust, this (among other things) has motivated an in-depth rewrite of the code in the form of Xranklin.jl [1] so I'd be curious to see if it works with it but fair enough if you don't have time to give it a shot

([1] it's the next version of Franklin which will be merged in some future and is available now if you want to try it out at Xranklin.jl)

[joa-quim]

Yes, the scatter! exists and works fine when called directly. It's when it (and others like it) is called nested (example image(G, scatter=(...)) that it takes the ! form and fails when run from Franklin, but only when the run is remote. Locally it works fine too. This is the mysterious part.

I already maintain to locations for the docs. For the first one I copy _site to the site (ignore the certificate warnings. Something to fix but it's so boring to learn how-to),

https://fct-gmt.ualg.pt/GMTjl_doc/examples/plotting_functions/05_1_stats/#example_17927783038414841698

and the second is what the GH action do. Note that this second version has several failed commands that show up fine is the fct-gmt.ualg.pt

https://www.generic-mapping-tools.org/GMTjl_doc/examples/plotting_functions/05_1_stats/#example_2653022517396184860

So you are saying that with optimize and publish I could set up a solution where I manually could upload to the gh-pages? That would be fine and I actually already tried to do so but failed miserably.

I also gave a look at Xranklin.jl but because the activity on it has not been much these last months I didn't go to the point of trying it. Will find some time to do so.

Thanks.

[tlienart]

So you are saying that with optimize and publish I could set up a solution where I manually could upload to the gh-pages? That would be fine and I actually already tried to do so but failed miserably.

yes that's right

I also gave a look at Xranklin.jl but because the activity on it has not been much these last months I didn't go to the point of trying it. Will find some time to do so.

Xranklin is basically done, just needs more people using it to do some polishing; I've not announced it because I haven't found the time to complete the docs (moved city, have a toddler, and started a company so quite a lot happening.. 😂 ); if you do end up giving it a shot, don't hesitate to open issues there, I do like replying and trying to figure things out with users and it will motivate me to get back at it when I have some spare time :)

[joa-quim]

So you are saying that with optimize and publish I could set up a solution where I manually could upload to the gh-pages? That would be fine and I actually already tried to do so but failed miserably.

yes that's right

Sorry to resurrect this, but how do I do that?

Don't know what is happening but nothing works anymore on Gh-Actions side anymore. Creating on Mac permanently hangs or crashes, on Linux there is a bunch of crazy errors, on Windows it doesn't work because of the rsync shit. And honestly I'm lacking courage (and time) to dive in Xranklin (it took me a lot of tame to get a working Franklin solution).

Thanks

[tlienart]

First remove your deployment workflow so that GA doesn't try to override what we do next (i.e. remove your .github/workflows/deploy.yml), commit and push

Approach 1 (with gh pages)

• optimize(prepath="GMTjl_doc", minify=false, prerender=false)
• copy content of __site to gh-pages branch and push

details

1. generate the site locally, check things are fine in a local browser
2. shut down the server, call optimize(prepath="GMTjl_doc", minify=false, prerender=false) (the options are not necessary but will keep stuff simple)
3. go to __site/index.html and check that your stylesheets have the proper path (e.g. <link rel="stylesheet" href="/GMTjl_doc/libs/highlight/github.min.css">)

assuming all went well so far, you want to push this to gh-pages and replace what was there, there's multiple ways to do this, the one below is one simple one.

1. copy __site somewhere else (e.g. your desktop)
2. in your local folder git checkout gh-pages && git pull
3. copy paste the content of __site to the folder replacing what's there & push these changes
4. checkout master/main
5. your content should be deployed within a couple of minutes

Approach 2 (without gh-pages)

If you don't like this because it's too manual, an alternative is to do the same stuff in your master branch, just ignore the gh-pages branch entirely, and in the github repository settings, for the github pages, select the branch master and select the folder __site as source). I.e. something like the screenshot below except you'd select __site instead of docs/ and master instead of main

Then you can just call

publish(prepath="GMTjl_doc", minify=false, prerender=false)

this will call optimize (to set the path correctly) then do a git push.

Note: make sure you don't have __site in your .gitignore if you decide to follow this approach.

Recommendations

Both approaches are probably not what you'd like to do unfortunately it's far easier to use github action to generate the website but this assumes you can use a linux runner which isn't your case.

• Approach 1 is preferable in the sense that you have fewer chances of messing up. It's the one I would recommend you use. You might want to write a simple script that does the steps for you.
• Approach 2 may seem easier and faster but if you apply changes on your site and then just do a git push you'll get a broken website because you'll have forgotten to call optimize which fixes your paths (e.g. to your stylesheets). Use this approach only if you're confident that you will never forget to call either
  a. optimize then the git push
  b. publish

[joa-quim]

Thanks a lot. At least I have something that works now. The option 1. worked but since __site is not under version control I had to manually git add many files (mostly figures) that were otherwise missing. But a problem I envisage with this is that since I'm also committing locally my .git directory will grow enormously.

I would certainly love to use GA only but don't understand what happens in that land. If I use a Linux builder there are tons of incomprehensible errors. Using the MacOS is a lottery where most of the times it hangs or crashes. Only yesterday I was able to have a almost decent build. Windows works fine but that damn JamesIves/github-pages-deploy-action@releases/v4 action is broken because it generates a filename with Windows path and the whole process really dies on the beach. I don't understand their pointer to the alternative solution https://github.com/JamesIves/github-pages-deploy-action#operating-system-support-
Tried it but my deploy.yml must had some (silent) errors because the build process didn't even start.

[tlienart]

Have a look at https://github.com/tlienart/test-fpush

• the workflow is in two parts, the first part could be windows, second should remain ubuntu
• it uploads __site as artifact, downloads it on (in this case) gh-preview and pushes

you could copy paste this changing gh-preview for gh-pages and that's basically a github action that does option 1 for you.

You can see here:

https://github.com/tlienart/test-fpush/actions/runs/3882968820

1 = franklin build
2 = download artifact, push to branch

Notes

• change prepath= @ https://github.com/tlienart/test-fpush/blob/a80e0e46bd070408600ca185238bbd0bee5391b4/.github/workflows/Deploy.yml#L21
• change gh-preview for whatever branch name you want to use for deploy (the branch doesn't have to exist) https://github.com/tlienart/test-fpush/blob/a80e0e46bd070408600ca185238bbd0bee5391b4/.github/workflows/Deploy.yml#L43
• change gh-preview https://github.com/tlienart/test-fpush/blob/a80e0e46bd070408600ca185238bbd0bee5391b4/.github/workflows/Deploy.yml#L57

[joa-quim]

Seems easy to adapt to my case. I'll try it next time I will need to update the manual.
Thanks for all the work it dedicated to this.

[joa-quim]

This thing looks cursed.

https://github.com/joa-quim/GMTjl_doc/actions/runs/3955082305/jobs/6773282818

Now fails again near the very end and with no explanation. But I see an error further up that I don't understand.
https://github.com/joa-quim/GMTjl_doc/actions/runs/3955082305/jobs/6773282818#step:8:718

[tlienart]

could you add me as collaborator?

[joa-quim]

You already are. That's the repo I kept for experiences like this. Tried also i the official one and got exactly the same error

https://github.com/GenericMappingTools/GMTjl_doc/actions/runs/3958861369/jobs/6781704426