On Mon, Jan 18, 2021 at 04:25:09PM +0100, Pierre-Yves Chibon wrote:
Good Morning Everyone,
...snip...
Currently our ideas are:
- for datanommer:
- port it to fedora-messaging
- adjust it to whichever solution we chose to replace datagrepper
- for datagrepper:
- keep it as is
- Replace by
- postgres
https://postgrest.org/
- prest
https://github.com/prest/prest
- kinto
https://docs.kinto-storage.org/en/stable/
- Swagger/OpenAPI
https://swagger.io/
Doing any of those means existing queries no longer work right?
Thats kind of a pain. ;(
- Add support for Graphql
- for the postgresql server
- Split messages per year in different table
- Unite them using a postgresql view
I've long wanted to do this. ;)
It might be worth making it non default to query more than the most
recent year. Most queries won't need anything that old...
- Kick out the old messages per year
- Keep the current year + n-1 in the current DB
- Kick the other to another DB?
- Kick the other to a tarball somewhere?
I would prefer to keep it queryable... there's some things that may want
to query the entire backhistory.
If you do this would you convert all the old messages?
Would you have any other ideas of things we could look at?
I'm not sure if there's any compression we could use here, but it would
be nice if the data took up less room. :)
Thanks for looking into this!
kevin