Re: Network setup of builders
by Rex Dieter
On 06/26/2013 02:25 PM, Miroslav Suchy wrote:
> On 06/26/2013 04:53 PM, seth vidal wrote:
>> 1. remember that the user can add random repos to their mock configs in
>> addition to the urls for their own srpms. Those repos can be from
>> anywhere - not just from coprs. This is on purpose - so we can support
>> a wide variety of systems and packages - w/ buildreqs from all over.
>
> Why we have such requirement?
Per first paragraph of http://fedoraproject.org/wiki/Category:Copr
"... to help make building and managing third party package repositories
*easy*." (emphasis mine).
Without this feature, copr's would lose a lot of what makes this easy.
-- rex
10 years, 9 months
Discussion: Using copr for building SCLs
by Bohuslav Kabrda
Hi list,
I'd like to start yet another discussion about some functionality of copr, here it is:
One of nice use cases for copr would be building software collections [1]. These however require modified mock chroot (modified config_opts['root'] and config_opts['chroot_setup_cmd']).
I'm trying to think of a way how to do this:
- We would need to allow users to create their own chroots (or more precisely allow them to do certain modifications).
- We would need to be able to send these chroots from frontend to backend.
- We would need some mechanism for sane naming, so that we can distinguish between builds of collections for different releases and architectures. E.g. ruby193-x86_64 is not enough, we need to also store the information about os name (epel/fedora) and os version (epel 6, fedora 18, ...).
My proposal:
- Use chroot names like ruby193_fedora-18-x86_64.
- Allow users to create these chroots by modifying config_opts['chroot_setup_cmd'] (config_opts['root'] would be autogenerated from the chroot name).
- Store the modified information in the MockChroot table and create these on backend using communication channel proposed in [2] (parts of it already implemented in frontend).
- Allow selecting these on copr creation/modification as more options for mock chroots, by default hidden under some kind of fancy clickable JS dropdown.
Thoughts?
--
Regards,
Bohuslav "Slavek" Kabrda.
[1] https://fedorahosted.org/SoftwareCollections/
[2] https://fedorahosted.org/copr/ticket/8
10 years, 9 months
[copr] master: add python-flexmock to deps of frontend (2c30fb0)
by Miroslav Suchý
Repository : http://git.fedorahosted.org/cgit/copr.git
On branch : master
>---------------------------------------------------------------
commit 2c30fb0d407ea533f7a8921f90bbed051b7311bd
Author: Miroslav Suchý <msuchy(a)redhat.com>
Date: Tue Jun 25 14:43:50 2013 +0200
add python-flexmock to deps of frontend
addressing:
_____________________________________ ERROR collecting tests/test_views/test_coprs_ns/test_coprs_general.py ______________________________________
tests/test_views/test_coprs_ns/test_coprs_general.py:4: in <module>
> from flexmock import flexmock
E ImportError: No module named flexmock
>---------------------------------------------------------------
copr.spec | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/copr.spec b/copr.spec
index 8ac71c8..19c40f4 100644
--- a/copr.spec
+++ b/copr.spec
@@ -58,6 +58,7 @@ Requires: python-psycopg2
Requires: python-whoosh
# for tests:
Requires: pytest
+Requires: python-flexmock
%description frontend
COPR is lightway Koji. It allows you to create new project in WebUI, and
10 years, 9 months
[copr] master: remove sentence which does not have meaning (3870790)
by Miroslav Suchý
Repository : http://git.fedorahosted.org/cgit/copr.git
On branch : master
>---------------------------------------------------------------
commit 38707907e468d7314ad27056934576e0d1dee5c0
Author: Miroslav Suchý <msuchy(a)redhat.com>
Date: Tue Jun 25 13:56:55 2013 +0200
remove sentence which does not have meaning
"you are you" does not have sense. I tried to rephrase it, but it is probably better without this sentece.
I think we do not need to explain why token expires.
>---------------------------------------------------------------
coprs_frontend/coprs/templates/api.html | 3 +--
1 files changed, 1 insertions(+), 2 deletions(-)
diff --git a/coprs_frontend/coprs/templates/api.html b/coprs_frontend/coprs/templates/api.html
index c57e49f..6501147 100644
--- a/coprs_frontend/coprs/templates/api.html
+++ b/coprs_frontend/coprs/templates/api.html
@@ -13,8 +13,7 @@
should not be shared!</span>.
</p>
- <p>The API token is valid for {{ config['API_TOKEN_EXPIRATION'] }} days after you generated it, this
- in order to ensure you are you every {{ config['API_TOKEN_EXPIRATION'] }} days.
+ <p>The API token is valid for {{ config['API_TOKEN_EXPIRATION'] }} days after you generated it.
</p>
{% if g.user %}
10 years, 9 months
[copr] master: change api token expiration to 120 days and make it configurable (171dd27)
by Miroslav Suchý
Repository : http://git.fedorahosted.org/cgit/copr.git
On branch : master
>---------------------------------------------------------------
commit 171dd27b1dd78072766f3828975f78226de20049
Author: Miroslav Suchý <msuchy(a)redhat.com>
Date: Tue Jun 25 13:50:59 2013 +0200
change api token expiration to 120 days and make it configurable
>---------------------------------------------------------------
coprs_frontend/coprs/templates/api.html | 4 ++--
coprs_frontend/coprs/views/api_ns/api_general.py | 2 +-
coprs_frontend/coprs/views/misc.py | 2 +-
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/coprs_frontend/coprs/templates/api.html b/coprs_frontend/coprs/templates/api.html
index 0ea30b2..c57e49f 100644
--- a/coprs_frontend/coprs/templates/api.html
+++ b/coprs_frontend/coprs/templates/api.html
@@ -13,8 +13,8 @@
should not be shared!</span>.
</p>
- <p>The API token is valid for 30 days after you generated it, this
- in order to ensure you are you every 30 days.
+ <p>The API token is valid for {{ config['API_TOKEN_EXPIRATION'] }} days after you generated it, this
+ in order to ensure you are you every {{ config['API_TOKEN_EXPIRATION'] }} days.
</p>
{% if g.user %}
diff --git a/coprs_frontend/coprs/views/api_ns/api_general.py b/coprs_frontend/coprs/views/api_ns/api_general.py
index 6d90033..7130509 100644
--- a/coprs_frontend/coprs/views/api_ns/api_general.py
+++ b/coprs_frontend/coprs/views/api_ns/api_general.py
@@ -38,7 +38,7 @@ def api_new_token():
user.api_token = helpers.generate_api_token(
flask.current_app.config['API_TOKEN_LENGTH'])
user.api_token_expiration = datetime.date.today() \
- + datetime.timedelta(days=30)
+ + datetime.timedelta(days=flask.current_app.config['API_TOKEN_EXPIRATION'])
db.session.add(user)
db.session.commit()
return flask.redirect(flask.url_for('api_ns.api_home'))
diff --git a/coprs_frontend/coprs/views/misc.py b/coprs_frontend/coprs/views/misc.py
index 7ffad6b..ea79640 100644
--- a/coprs_frontend/coprs/views/misc.py
+++ b/coprs_frontend/coprs/views/misc.py
@@ -56,7 +56,7 @@ def create_or_login(resp):
models.User.openid_name == resp.identity_url).first()
if not user: # create if not created already
expiration_date_token = datetime.date.today() \
- + datetime.timedelta(days=30)
+ + datetime.timedelta(days=flask.current_app.config['API_TOKEN_EXPIRATION'])
copr64 = base64.b64encode('copr') + '##'
user = models.User(openid_name = resp.identity_url, mail = resp.email,
api_login = copr64 + helpers.generate_api_token(
10 years, 9 months
refactoring backend build spawning
by Seth Vidal
I've been noodling around with how to allow for arbitrary things to be
run prior to the actual building of the pkgs. This is to allow for a
wide range of 'sources' of pkgs.
I talked with Slavek at summit briefly and we talked about a way to
handle the giturl-of-doom or turning various arbitrary blobs into srpms.
The gist of it is this - we have a place we can do it in the code we
have but we will need to change how builds are being spawned.
Right now we just a direct ansible call to execute mockchain and get
the srpms building. This is simple and immediate but it means we have
to modify code every time we want to add another step.
I'm thinking we use a playbook instead of a direct call. This helps us
b/c it means the playbook is easier to modify and we can get the log
results out of its objects.
for anyone not familiar with ansible playbooks see here:
http://www.ansibleworks.com/docs/playbooks.html
Here's is the framework I was thinking of:
playbook:
play
- setup/create builder
- provision it
- add mock configs, etc
play
- verify state of the builder
play
- download srpms/pkgs/giturls/etc
- make into srpms (optionally)
- check integrity of srpms
- check licenses - essentially run the rpmlint license checker
- attempt buildorder sort? (optional but could be handy for a number of
cases - this is using this code:
http://skvidal.fedorapeople.org/misc/buildorder/
- mockchain build srpms
- async (perhaps to make it easier to timeout - harder to retrieve
logs)
play
- download results
play
- destroy builder
This playbook would let us modify the playbook for the type of system
(for example the mechanism to spin up an arm-builder is very very
different from an x86[_64] vm.
The way I figure this would allow us to insert a set of random tasks
into the entire build process w/o having to add more code - just add
more plays into the playbook.
thoughts?
-sv
10 years, 9 months
[copr] master: create_chroot must be run as copr-fe user (76a1199)
by Miroslav Suchý
Repository : http://git.fedorahosted.org/cgit/copr.git
On branch : master
>---------------------------------------------------------------
commit 76a1199e18746b4239f571fe71415585f0d07529
Author: Miroslav Suchý <msuchy(a)redhat.com>
Date: Mon Jun 24 16:47:03 2013 +0200
create_chroot must be run as copr-fe user
addressing:
mod_wsgi (pid=1279): Exception occurred processing WSGI script '/usr/share/copr/coprs_frontend/application'.
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/flask/app.py", line 1701, in __call__
return self.wsgi_app(environ, start_response)
File "/usr/lib/python2.7/site-packages/flask/app.py", line 1689, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/usr/lib/python2.7/site-packages/flask/app.py", line 1687, in wsgi_app
response = self.full_dispatch_request()
File "/usr/lib/python2.7/site-packages/flask/app.py", line 1360, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/lib/python2.7/site-packages/flask/app.py", line 1358, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/lib/python2.7/site-packages/flask/app.py", line 1344, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/share/copr/coprs_frontend/coprs/views/misc.py", line 120, in decorated_function
return f(*args, **kwargs)
File "/usr/share/copr/coprs_frontend/coprs/views/coprs_ns/coprs_general.py", line 89, in copr_new
db.session.commit()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/scoping.py", line 149, in do
return getattr(self.registry(), name)(*args, **kwargs)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 720, in commit
self.transaction.commit()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 360, in commit
self.session.dispatch.after_commit(self.session)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/event.py", line 409, in __call__
fn(*args, **kw)
File "/usr/lib/python2.7/site-packages/flask_sqlalchemy.py", line 185, in after_commit
models_committed.send(session.app, changes=d.values())
File "/usr/lib/python2.7/site-packages/blinker/base.py", line 220, in send
for receiver in self.receivers_for(sender)]
File "/usr/lib/python2.7/site-packages/flask_whooshee.py", line 174, in on_commit
writer = wh.index.writer()
File "/usr/lib/python2.7/site-packages/whoosh/filedb/fileindex.py", line 258, in writer
return SegmentWriter(self, **kwargs)
File "/usr/lib/python2.7/site-packages/whoosh/filedb/filewriting.py", line 136, in __init__
delay=delay):
File "/usr/lib/python2.7/site-packages/whoosh/support/filelock.py", line 54, in try_for
v = fn()
File "/usr/lib/python2.7/site-packages/whoosh/support/filelock.py", line 98, in acquire
self.fd = os.open(self.filename, flags)
OSError: [Errno 13] Permission denied: '/usr/share/copr/data/whooshee/copr_user_whoosheer/MAIN_WRITELOCK'
>---------------------------------------------------------------
copr-setup.txt | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/copr-setup.txt b/copr-setup.txt
index 1e12c30..f5a7ad6 100644
--- a/copr-setup.txt
+++ b/copr-setup.txt
@@ -83,7 +83,7 @@ COPRS_ENVIRON_PRODUCTION=1 ./manage.py create_db --alembic alembic.ini
- setenforce 0 (sorry, putting on TODO)
- Add chroots. E.g:
-COPRS_ENVIRON_PRODUCTION=1 ./manage.py create_chroot fedora-18-x86_64 fedora-18-i386
+ su - copr-fe -c 'OPRS_ENVIRON_PRODUCTION=1 ./manage.py create_chroot fedora-18-x86_64 fedora-18-i386'
DB config
10 years, 9 months
[copr] master: add note that you have to add chroots to db (cca7c7c)
by Miroslav Suchý
Repository : http://git.fedorahosted.org/cgit/copr.git
On branch : master
>---------------------------------------------------------------
commit cca7c7ce8609a1e3606dfe07f615b944f9d2843a
Author: Miroslav Suchý <msuchy(a)redhat.com>
Date: Mon Jun 24 16:23:32 2013 +0200
add note that you have to add chroots to db
>---------------------------------------------------------------
copr-setup.txt | 4 ++++
1 files changed, 4 insertions(+), 0 deletions(-)
diff --git a/copr-setup.txt b/copr-setup.txt
index a3dd962..1e12c30 100644
--- a/copr-setup.txt
+++ b/copr-setup.txt
@@ -82,6 +82,10 @@ COPRS_ENVIRON_PRODUCTION=1 ./manage.py create_db --alembic alembic.ini
- setenforce 0 (sorry, putting on TODO)
+- Add chroots. E.g:
+COPRS_ENVIRON_PRODUCTION=1 ./manage.py create_chroot fedora-18-x86_64 fedora-18-i386
+
+
DB config
=========
10 years, 9 months
[copr] master: mark config.py as config so it is not overwritten during upgrade (3232414)
by Miroslav Suchý
Repository : http://git.fedorahosted.org/cgit/copr.git
On branch : master
>---------------------------------------------------------------
commit 3232414e6066a62e0c4a9f6f2713dc45a84783a3
Author: Miroslav Suchý <msuchy(a)redhat.com>
Date: Mon Jun 24 16:18:09 2013 +0200
mark config.py as config so it is not overwritten during upgrade
>---------------------------------------------------------------
copr.spec | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/copr.spec b/copr.spec
index dd80c51..8ac71c8 100644
--- a/copr.spec
+++ b/copr.spec
@@ -136,6 +136,7 @@ useradd -r -g copr-fe -G copr-fe -d %{_datadir}/copr/coprs_frontend -s /bin/bash
%{_datadir}/copr/coprs_frontend
%ghost %{_datadir}/copr/data/copr.db
+%config(noreplace)%{_datadir}/copr/coprs_frontend/coprs/config.py
%files cli
%doc LICENSE README.rst
10 years, 9 months