Cross posting this in ruby-sig since this concerns ruby packaging and in devel since I have some more generic questions. Excuse me if I created unecessary noise. For those not familiar with the project, you'll find some info here [http://axilleas.me/tag/gsoc.html].
Let me start with the directory structure of these packages. We have two locations: 1) where the main gitlab and gtlab-shell rails apps reside, 2) where git repositories and their satellites[0] are.
I have asked in #fedora-infra what FHS they use with the git repos in fedorahosted and we concluded that the rails apps would go to /usr/share/ and git repos and satellites to /usr/lib/. Now, when specifing %{_libdir} in a spec, it gets /usr/lib64/ on a x86_64 machine. I wouldn't want to install gitlab in different dirs when different arches, so for now I used /var/lib/ for that purpose. I guess I could excplicitly say /usr/lib/gitlab/ but anyway.
So the current structure is:
|-- /usr/share/gitlab/ | |-- gitlab/ | |-- gitlab-shell/ | |-- /var/lib/gitlab/ | |-- satellites/ | |-- repositories/ | |-- .ssh/authorized_keys | |-- /etc/gitlab/ | |-- gitlab.yml | |-- shell.yml | |-- database.yml | |-- unicorn.rb
In /etc there will be configuration files with symlinks in the rails app dirs. What are your thoughts on the directory locations? Do you agree?
And now my questions about the specs. FYI I have seen the katello.spec and took some info from there[1].
gitlab-shell.spec -----------------
This is kinda finished.
SPEC: https://github.com/axilleas/fedora/blob/master/packages/gitlab-shell/gitlab-...
I tested the package and the appropriate user and group are created when install/upgrade but when uninstalling, the homedir doesn't get removed. I suspect this has something to do with useradd and protecting the homedir of the user?
gitlab.spec -----------
This is a draft, I didn't test to install yet.
SPEC: https://github.com/axilleas/fedora/blob/master/packages/gitlab/gitlab.spec
## Ruby specific
- rake tasks
Many jobs, like the backup, initial database seed, etc. are done with rake tasks. How do we invoke them without getting in the app root dir every time? Is there some sort of mechanism for that?
- Are Gemfile and Gemfile.lock really needed for the app to work? If yes how can we avoid this? Using bundler_ext? If yes, how?
This is an interesting question as katello uses bundler_ext but I haven't yet understood how that works. Any tip would be appreciated. For now, in the test instance I set up, unicorn checks the Gemfile and if not found it doesn't spawn. Here is an entry in its log when I removed the Gemfile:
I, [2013-09-19T21:00:13.056391 #18224] INFO -- : Refreshing Gem list
and then an error occured.
## Generic
- symlink logs to /var/log/gitlab/
Not all logs' directory is configurable. GitLab uses the logs found in app_dir/log/ and displays them via its ui to the admin users. Should I symlink these logs to /var/log/ for ease of access?
- pids: move to /var/run/gitlab/ (?)
GitLab is practically running using unicorn and sidekiq. These two create each its own pid file in app_dir/tmp/pids/ by default. Luckily this is configurable via their configs or systemd services. Also unicorn creates a gitlab.socket which uses to speak with the app. If we use apache this isn't needed, but with nginx we can use it. I was thinking it could go under /var/run/gitlab/ too.
- how to support both databases. Is it feasible?
GitLab supports mysql(mariadb for us) and postgres. How do we deal with these cases inside a spec file? For now, I have added a comment about the postgres config and made mariadb the default one.
- ownership of directories
In upstream installation guide, gitlab and gitlab-shell reside in the same location that's why I decided to have them both under /usr/share/gitlab/. And my question is, which package owns /usr/share/gitlab/? Both?
Thanks for reading, it took me quite a few hours to compose :)
[0] satellites is a directory where it is stored information about the merge requests, forks, etc of each git repository. [1] https://github.com/axilleas/fedora/blob/master/packages/SPECS/katello.spec
Dne 19.9.2013 20:42, Axilleas Pipinellis napsal(a):
I have asked in #fedora-infra what FHS they use with the git repos in fedorahosted and we concluded that the rails apps would go to /usr/share/ and git repos and satellites to /usr/lib/.
Git repos in /usr/lib? That does not sound right. Later, you mention on same places /var/lib, that is more appropriate IMO. You might want to link to specific section of FHS and elaborate on such decision.
So the current structure is:
|-- /usr/share/gitlab/ | |-- gitlab/ | |-- gitlab-shell/ | |-- /var/lib/gitlab/ | |-- satellites/ | |-- repositories/ | |-- .ssh/authorized_keys | |-- /etc/gitlab/ | |-- gitlab.yml | |-- shell.yml | |-- database.yml | |-- unicorn.rb
In /etc there will be configuration files with symlinks in the rails app dirs. What are your thoughts on the directory locations? Do you agree?
That looks good. However, I am not sure about the /etc/gitlab. Why there should be linked all the configuration? Current trend (which I support) is keep default configuration somewhere by the application, e.g. in /usr/{lib,share} and into /etc/gitlab place just configuration overrides, i.e. if you need to differ, you place configuration file into /etc/gitlab, otherwise the defaults are taken.
And now my questions about the specs. FYI I have seen the katello.spec and took some info from there[1].
gitlab-shell.spec
This is kinda finished.
SPEC: https://github.com/axilleas/fedora/blob/master/packages/gitlab-shell/gitlab-...
I tested the package and the appropriate user and group are created when install/upgrade but when uninstalling, the homedir doesn't get removed. I suspect this has something to do with useradd and protecting the homedir of the user?
gitlab.spec
This is a draft, I didn't test to install yet.
SPEC: https://github.com/axilleas/fedora/blob/master/packages/gitlab/gitlab.spec
## Ruby specific
- rake tasks
Many jobs, like the backup, initial database seed, etc. are done with rake tasks. How do we invoke them without getting in the app root dir every time? Is there some sort of mechanism for that?
You can call "rake -f /path/to/your/Rakefile", but it depends how the task is written.
## Generic
- symlink logs to /var/log/gitlab/
Not all logs' directory is configurable.
They should be. Logging somewhere into /usr/share makes no sense.
- pids: move to /var/run/gitlab/ (?)
GitLab is practically running using unicorn and sidekiq. These two create each its own pid file in app_dir/tmp/pids/ by default. Luckily this is configurable via their configs or systemd services. Also unicorn creates a gitlab.socket which uses to speak with the app. If we use apache this isn't needed, but with nginx we can use it. I was thinking it could go under /var/run/gitlab/ too.
Sounds right.
- how to support both databases. Is it feasible?
GitLab supports mysql(mariadb for us) and postgres. How do we deal with these cases inside a spec file? For now, I have added a comment about the postgres config and made mariadb the default one.
I would go with something like 'gitlab-mariadb' and 'gitlab-postgres', which would provide appropriate configurations, but some could argue, that configuring system by installation of package is wrong.
- ownership of directories
In upstream installation guide, gitlab and gitlab-shell reside in the same location that's why I decided to have them both under /usr/share/gitlab/. And my question is, which package owns /usr/share/gitlab/? Both?
Both, if they are independent. Or you can create some -filesystem package, which would own the directory.
Vít
On 09/23/2013 02:11 PM, Vít Ondruch wrote:
Dne 19.9.2013 20:42, Axilleas Pipinellis napsal(a):
I have asked in #fedora-infra what FHS they use with the git repos in fedorahosted and we concluded that the rails apps would go to /usr/share/ and git repos and satellites to /usr/lib/.
Git repos in /usr/lib? That does not sound right. Later, you mention on same places /var/lib, that is more appropriate IMO. You might want to link to specific section of FHS and elaborate on such decision.
/usr/lib/ was the outcome of a brief discussion in #fedora-admin (sorry, in my previous message I wrote #fedora-infra). I then changed it to /var/lib/ as it seemed more appropriate.
So the current structure is:
|-- /usr/share/gitlab/ | |-- gitlab/ | |-- gitlab-shell/ | |-- /var/lib/gitlab/ | |-- satellites/ | |-- repositories/ | |-- .ssh/authorized_keys | |-- /etc/gitlab/ | |-- gitlab.yml | |-- shell.yml | |-- database.yml | |-- unicorn.rb
In /etc there will be configuration files with symlinks in the rails app dirs. What are your thoughts on the directory locations? Do you agree?
That looks good. However, I am not sure about the /etc/gitlab. Why there should be linked all the configuration? Current trend (which I support) is keep default configuration somewhere by the application, e.g. in /usr/{lib,share} and into /etc/gitlab place just configuration overrides, i.e. if you need to differ, you place configuration file into /etc/gitlab, otherwise the defaults are taken.
I'm not sure how "configuration overrides" could be supported. In /etc/gitlab/ all configs are symlinked to the relevant app dir so an admin can change them easily.
- rake tasks
Many jobs, like the backup, initial database seed, etc. are done with rake tasks. How do we invoke them without getting in the app root dir every time? Is there some sort of mechanism for that?
You can call "rake -f /path/to/your/Rakefile", but it depends how the task is written.
Since the app will be owned by a separate user, I was thinking something like that could pull it off:
su - gitlab -s /bin/sh -c "cd gitlab; /usr/bin/rake db:migrate RAILS_ENV=production"
Or we could build a script that would take rake's commands as arguments.
## Generic
- symlink logs to /var/log/gitlab/
Not all logs' directory is configurable.
They should be. Logging somewhere into /usr/share makes no sense.
I will talk about this with the devs.
- pids: move to /var/run/gitlab/ (?)
GitLab is practically running using unicorn and sidekiq. These two create each its own pid file in app_dir/tmp/pids/ by default. Luckily this is configurable via their configs or systemd services. Also unicorn creates a gitlab.socket which uses to speak with the app. If we use apache this isn't needed, but with nginx we can use it. I was thinking it could go under /var/run/gitlab/ too.
Sounds right.
Done.
- how to support both databases. Is it feasible?
GitLab supports mysql(mariadb for us) and postgres. How do we deal with these cases inside a spec file? For now, I have added a comment about the postgres config and made mariadb the default one.
I would go with something like 'gitlab-mariadb' and 'gitlab-postgres', which would provide appropriate configurations, but some could argue, that configuring system by installation of package is wrong.
I was thinking that we just require both rubygem-mysql and rubygem-pg in spec. Then one could just change the /etc/gitlab/database.yml to his/her liking (although one of them will be the default).
- ownership of directories
In upstream installation guide, gitlab and gitlab-shell reside in the same location that's why I decided to have them both under /usr/share/gitlab/. And my question is, which package owns /usr/share/gitlab/? Both?
Both, if they are independent. Or you can create some -filesystem package, which would own the directory.
And by independent you mean they reside in separate dirs, right? If so then yes, they are independent.
ruby-sig@lists.fedoraproject.org
-
Achilleas Pipinellis -
Vít Ondruch