Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance documentation to discuss database updates #74

Open
marcstern opened this issue Nov 23, 2018 · 5 comments
Open

Enhance documentation to discuss database updates #74

marcstern opened this issue Nov 23, 2018 · 5 comments

Comments

@marcstern
Copy link

The documentation doesn't say if httpd must be stopped when updating the DB.
That should be added: no restart, restart, stop/start?

@horgh
Copy link
Contributor

horgh commented Nov 24, 2018

Yeah, I agree it would be useful to say something about this in the docs.

Reloading should be sufficient to see the new version.

@marcstern
Copy link
Author

Parts of the DB are never reread from disk, even in case of swapping, etc. Otherwise, a running process could get some data about the new version while still having parts of the old one in memory.

@horgh
Copy link
Contributor

horgh commented Nov 26, 2018

It shouldn't be the case that you have two conflicting versions of the database in use like you describe.

How are you updating the database? The files get memory mapped, so you'll want to be sure that when you update them you do not rewrite the files (e.g. with cp) but instead replace them (e.g. with mv on the same filesystem).

@oschwald oschwald changed the title DB update Enhance documentation to discuss database updates Aug 21, 2019
@h0tw1r3
Copy link

h0tw1r3 commented Apr 21, 2021

Making a note which I hope is picked up by search bots.

If you have a process that updates all your servers from an internal mirror, something like:

/usr/bin/wget -q -r -nd -N -A '*.mmdb' -P /usr/share/GeoIP http://mirror.lan/share/geoip/

If there are changes, the mmdb file is rewritten. On a very active server, almost immediately your logs will be overwhelmed with error messages like these:

[mod_maxminddb]: Error getting data for '1.2.3.4': The MaxMind DB file's data section contains bad data (unknown data type or corrupt data)

Reloading the server will "fix" the problem, but the time between the update and the reload anything that relies on the environment variables is broken.
As @horgh suggests, replacing (mv vs cp) should also work around the issue, but in a very active system there's still room for error.

I wonder if implementing the CheckCache, MemoryCache, and IndexCache options from mod_geoip2 would be worth the trouble.

@oschwald
Copy link
Member

Even with those options, the database would be corrupt while it is being copied over. You would need to atomically replace the database file, e.g., with mv on the same file system, to prevent the possibility of errors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

4 participants