Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migration from MediaWiki #2

Open
1 of 2 tasks
nickswalker opened this issue Oct 11, 2017 · 0 comments
Open
1 of 2 tasks

Migration from MediaWiki #2

nickswalker opened this issue Oct 11, 2017 · 0 comments

Comments

@nickswalker
Copy link
Member

nickswalker commented Oct 11, 2017

After some discussion in BWI meetings, we agreed that we should move towards hosting the Wiki-style documentation for the project on GitHub. The current MediaWiki installation requires us to make a separate for every contributor and barrier to access has contributed to the wiki's neglect. There's also useful information about the project that isn't accessible to outside groups due to the login requirement.

I have migrated the installation by following these steps to export pages from a Wiki and processing the output through a MediaWiki to Markdown converter. I did some post processing on the output to make the output suitable for GitHub (mostly to address this issue philipashlock/mediawiki-to-markdown#12, see scripts at end of this comment). I left out pages that were only about old FRI class projects or that were primarily about the administration of the BWI lab machines. I removed all references to passwords. I pushed the results to this repo's wiki sub-repo.

There are a couple of things left before the migration is complete:

  • Create a place to host the private documents. utexas-bwi/documentation-private?
  • Migrate images. Some pages like those about CMASS have some nice diagrams and images that we should keep. I think this will have to be manual

Post processing (zsh on macOS):

# Make links local
sed -i.bak 's/\](\//\](\.\//g' **/*.md

# Remove "wikilink" note
sed -i.bak 's/[[:space:]]"wikilink"//g' **/*.md

# Remove Category annotation
sed -i.bak 's/\[Category.*)//g' **/*.md

# Remove the front matter from each page
for filename in $(ls **/*.md); do
    # Remove the first four lines
    sed -i.bak -e '1,4d' $filename
done

for filename in $(ls **/*.md); do
	# delete all leading blank lines at top of file
	sed -i.bak -e '/./,$!d' $filename
done

# Change underscores to dashes in filenames
for filename in $(ls **/*.md); do
	mv -i "$filename" "$(dirname "$filename")/$(basename "$filename"|tr '_' '-')";
done

# Replace underscores in links that point to other wiki pages
for filename in $(ls **/*.md); do
	sed -i.bak -e 's/\(\]([\.]*\/[^_]*\)\(_\)\(.*\)/\1-\3/g' $filename
done

# Check for obvious secrets
grep -r "password" .
grep -r "account" .
grep -r "pass" .
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant