aeolus-devel April 2011

aeolus-devel@lists.fedorahosted.org

32 participants
122 discussions

#464/466 - puppet script configures pulp to sync f14/f13 repo
by Richard Su 26 Apr '11

26 Apr '11

This patch requires some setup. These manual steps will hopefully be eliminated once additional tasks in redmine are completed (1263, 1252, and 1262) 1. Install the pulp rpm. The pulp repo may need to be added. Instructions are here: http://pulpproject.org/ug/UGInstallation.html#installation 2. After pulp has been installed, you need to edit and replace line 182 of /usr/lib/python2.7/site-packages/pulp/client/server.py with: default_locale = locale.getdefaultlocale()[0] if default_locale: default_locale = default_locale.lower().replace('_', '-') else: default_locale = "us-en" This is a temporary workaround until https://bugzilla.redhat.com/show_bug.cgi?id=629718 has been fixed. 3. Finally, in /etc/pulp/client.conf, change host from localhost.localdomain to actual.

1 1

[PATCH configure] puppet pulp sync script v3
by Richard Su 26 Apr '11

26 Apr '11

offending whitespaces were removed from previous version 1. check pulp rpm is installed 2. write out templatized configuration files 3. start pulp-server service 4. initializes pulp 5. saves default credentials (may need to change admin password) 6. creates x86_64 repos for f13 and f14 7. triggers a sync on both repos There is an option to force the sync and wait until the sync completes before returning status to puppet. For me, the initial sync took 1hr and subsequent syncs took 2 minutes. I decided not to use the force option and let sync happen in the background. --- bin/aeolus-cleanup | 2 + bin/aeolus-configure | 2 + recipes/aeolus_recipe/aeolus_recipe.pp | 2 +- recipes/aeolus_recipe/aeolus_uninstall.pp | 2 +- recipes/aeolus_recipe/manifests/aeolus.pp | 9 ++- recipes/aeolus_recipe/manifests/conductor.pp | 2 +- recipes/aeolus_recipe/manifests/pulp.pp | 69 +++++++++++++ recipes/aeolus_recipe/templates/pulp-client.conf | 49 +++++++++ recipes/aeolus_recipe/templates/pulp.conf | 119 ++++++++++++++++++++++ recipes/apache/manifests/init.pp | 7 +- 10 files changed, 257 insertions(+), 6 deletions(-) create mode 100644 recipes/aeolus_recipe/manifests/pulp.pp create mode 100755 recipes/aeolus_recipe/templates/pulp-client.conf create mode 100644 recipes/aeolus_recipe/templates/pulp.conf diff --git a/bin/aeolus-cleanup b/bin/aeolus-cleanup index acf3fed..87fb405 100644 --- a/bin/aeolus-cleanup +++ b/bin/aeolus-cleanup @@ -1,7 +1,9 @@ #!/bin/sh +export FACTER_AEOLUS_ENABLE_HTTPS=true export FACTER_AEOLUS_ENABLE_SECURITY=false puppet /usr/share/aeolus-configure/aeolus_uninstall.pp \ --modulepath=/usr/share/aeolus-configure/modules/ \ --logdest=/var/log/aeolus-configure/aeolus-cleanup.log \ --logdest=console + diff --git a/bin/aeolus-configure b/bin/aeolus-configure index 34fa2ff..6d87cb1 100644 --- a/bin/aeolus-configure +++ b/bin/aeolus-configure @@ -1,7 +1,9 @@ #!/bin/sh +export FACTER_AEOLUS_ENABLE_HTTPS=true export FACTER_AEOLUS_ENABLE_SECURITY=false puppet /usr/share/aeolus-configure/aeolus_recipe.pp \ --modulepath=/usr/share/aeolus-configure/modules/ \ --logdest=/var/log/aeolus-configure/aeolus-configure.log \ --logdest=console + diff --git a/recipes/aeolus_recipe/aeolus_recipe.pp b/recipes/aeolus_recipe/aeolus_recipe.pp index dd05dc3..82b3e54 100644 --- a/recipes/aeolus_recipe/aeolus_recipe.pp +++ b/recipes/aeolus_recipe/aeolus_recipe.pp @@ -30,6 +30,7 @@ import "aeolus_recipe/defaults" include aeolus::conductor include aeolus::image-factory include aeolus::iwhd +include aeolus::pulp aeolus::create_bucket{"aeolus":} @@ -68,4 +69,3 @@ aeolus::conductor::hwp{"hwp1": login_user => 'admin', login_password => 'password', require => Aeolus::Site_admin["admin"] } - diff --git a/recipes/aeolus_recipe/aeolus_uninstall.pp b/recipes/aeolus_recipe/aeolus_uninstall.pp index c0b0eae..8dba827 100644 --- a/recipes/aeolus_recipe/aeolus_uninstall.pp +++ b/recipes/aeolus_recipe/aeolus_uninstall.pp @@ -30,8 +30,8 @@ import "aeolus_recipe/defaults" include aeolus::conductor::disabled include aeolus::iwhd::disabled include aeolus::image-factory::disabled +include aeolus::pulp::disabled aeolus::deltacloud::disabled{"mock": } aeolus::deltacloud::disabled{"ec2-us-east-1": } aeolus::deltacloud::disabled{"ec2-us-west-1": } - diff --git a/recipes/aeolus_recipe/manifests/aeolus.pp b/recipes/aeolus_recipe/manifests/aeolus.pp index 2b7d54e..d94ed38 100644 --- a/recipes/aeolus_recipe/manifests/aeolus.pp +++ b/recipes/aeolus_recipe/manifests/aeolus.pp @@ -10,6 +10,14 @@ import "conductor" import "deltacloud" import "iwhd" import "image-factory" +import "pulp" + +if $aeolus_enable_https == "true" or $aeolus_enable_https == "1" { + import "openssl" + $enable_https = true +} else { + $enable_https = false +} if $aeolus_enable_security == "true" or $aeolus_enable_security == "1" { import "openssl" @@ -56,4 +64,3 @@ define aeolus::provider($type, $port, $login_user="", $login_password=""){ login_password => $login_password, require => Aeolus::Deltacloud[$name] } } - diff --git a/recipes/aeolus_recipe/manifests/conductor.pp b/recipes/aeolus_recipe/manifests/conductor.pp index 4d0fe43..474424a 100644 --- a/recipes/aeolus_recipe/manifests/conductor.pp +++ b/recipes/aeolus_recipe/manifests/conductor.pp @@ -149,7 +149,7 @@ class aeolus::conductor inherits aeolus { ### Setup apache for deltacloud include apache - if $enable_security { + if $enable_https { apache::site{"aeolus-conductor": source => 'puppet:///modules/aeolus_recipe/aggregator-httpd-ssl.conf'} } else{ apache::site{"aeolus-conductor": source => 'puppet:///modules/aeolus_recipe/aggregator-httpd.conf'} diff --git a/recipes/aeolus_recipe/manifests/pulp.pp b/recipes/aeolus_recipe/manifests/pulp.pp new file mode 100644 index 0000000..0c5a198 --- /dev/null +++ b/recipes/aeolus_recipe/manifests/pulp.pp @@ -0,0 +1,69 @@ +# Aeolus pulp puppet definitions + +class aeolus::pulp inherits aeolus { + package { 'pulp': + ensure => 'installed' + } + + file { 'pulp-conf-file-pulp': + path => "/etc/pulp/pulp.conf", + owner => root, + group => root, + content => template("aeolus_recipe/pulp.conf") + } + + file { 'pulp-conf-file-client': + path => "/etc/pulp/client.conf", + owner => apache, + group => apache, + content => template("aeolus_recipe/pulp-client.conf") + } + + exec{'pulp-init': + command => "/sbin/service pulp-server init", + logoutput => true, + require => [Package['pulp'], + File['pulp-conf-file-pulp'], + File['pulp-conf-file-client']]} + + service { 'pulp-server': + ensure => 'running', + enable => true, + require => [Package['pulp'], + Exec["pulp-init"]]} + + exec{'pulp-auth': + command => "/usr/bin/pulp-admin -u admin -p admin auth login --username admin --password admin", + logoutput => true, + require => Service['pulp-server']} + + exec{'pulp-f13-x86-64-repo-create': + command => "/usr/bin/pulp-admin repo create --id f13-x86_64 --feed yum:http://download.fedoraproject.org/pub/fedora/linux/releases/13/Fedora/x…", + logoutput => true, + creates => "/var/lib/pulp/repos/pub/fedora/linux/releases/13/Fedora/x86_64/os", + require => Exec['pulp-auth']} + + exec{'pulp-f13-x86-64-repo-sync': + command => "/usr/bin/pulp-admin repo sync --id f13-x86_64", + logoutput => true, + require => Exec['pulp-f13-x86-64-repo-create']} + + exec{'pulp-f14-x86-64-repo-create': + command => "/usr/bin/pulp-admin repo create --id f14-x86_64 --feed yum:http://download.fedoraproject.org/pub/fedora/linux/releases/14/Fedora/x…", + logoutput => true, + creates => "/var/lib/pulp/repos/pub/fedora/linux/releases/14/Fedora/x86_64/os", + require => Exec['pulp-auth']} + + exec{'pulp-f14-x86-64-repo-sync': + command => "/usr/bin/pulp-admin repo sync --id f14-x86_64", + logoutput => true, + require => Exec['pulp-f14-x86-64-repo-create']} +} + +class aeolus::pulp::disabled { + service { 'pulp-server': + ensure => 'stopped', + enable => false, + hasstatus=> true + } +} diff --git a/recipes/aeolus_recipe/templates/pulp-client.conf b/recipes/aeolus_recipe/templates/pulp-client.conf new file mode 100755 index 0000000..7c990e8 --- /dev/null +++ b/recipes/aeolus_recipe/templates/pulp-client.conf @@ -0,0 +1,49 @@ +# The pulp server configuration +# host : The pulp server +# port : The port providing the RESTful API. +# scheme : The protocol. +# interval : The agent update interval +[server] +host = <%= fqdn %> +port = 443 +scheme = https +path = /pulp/api +interval = 240 + +# Client configuration +# reboot_schedule : Time in minutes or 'now' before system is scheduled for reboot +# when user requests a reboot for applicable errata. +# assumeyes : Uncomment this to override this value. Assumes that install performs all the suggested actions such as reboot on successful install; +# import_gpg_keys : automatically import gpg keys if available during remote package installs +[client] +reboot_schedule = 3 +import_gpg_keys = False +#assumeyes = False +repo_file = /etc/yum.repos.d/pulp.repo +mirror_list_dir = /etc/yum.repos.d +gpg_keys_dir = /etc/pki/pulp-gpg-keys + +# heartbeat +# interval in seconds +[heartbeat] +seconds=10 + +# messaging +# scheme: (tcp|ssl) +# port: broker port +# cacert: the CA certificate (PEM) to verify the server. +# clientcert: The client (PEM) key & certificate. +[messaging] +scheme = tcp +port = 5672 +cacert = +clientcert = + + +# Content distribution options +# baseurl : The content base url +# keyurl : The gpg key base url +[cds] +baseurl = https://localhost/pulp/repos +keyurl = http://localhost/pulp/gpg +ksurl = http://localhost/pulp/ks diff --git a/recipes/aeolus_recipe/templates/pulp.conf b/recipes/aeolus_recipe/templates/pulp.conf new file mode 100644 index 0000000..ef34eaf --- /dev/null +++ b/recipes/aeolus_recipe/templates/pulp.conf @@ -0,0 +1,119 @@ +# Server options +# +# server_name: server hostname(protocol defaults to https) +# relative_url: the relative url at which repos are exposed to the clients +# key_url: the relative url at which gpg keys are exposed to clients +# default_login: default admin username for your pulp server +# default_password: default password for admin +# Highly recommend changing the default_password with "pulp-admin user update" +# +[server] +server_name: <%= fqdn %> +relative_url: /pulp/repos +key_url: /pulp/gpg +default_login: admin +default_password: admin + +# Messaging options. +# +# url: the broker url. +# cacert: path to PEM encoded CA certificate file +# clientcert: path to PEM encoded file containing both +# the private key and certificate. +# +# The url has form: <transport>://<host>:<port> where +# transport can be (tcp|ssl). When ssl is specified, +# cacert and clientcert must be specified. + +[messaging] +url: tcp://localhost:5672 +cacert: /etc/pki/qpid/ca/ca.crt +clientcert: /etc/pki/qpid/client/client.pem + +# +# AMQP event processing +# +[events] +send_enabled: false +recv_enabled: false + +# Configures aspects of the pulp web server security. +# +# cacert: full path to the CA certificate that will be used to sign +# consumer and admin identification certificates. This MUST match +# the value of SSLCACertificateFile in /etc/httpd/conf.d/pulp.conf. +# cakey: full path to the private key for the CA certificate +# + +[security] +cacert: /etc/pki/pulp/ca.crt +cakey: /etc/pki/pulp/ca.key +# oauth_key: string key to enable OAuth style authentication +# oauth_secret: string shared secret that can be used for OAuth style authentication +# For more info see: https://fedorahosted.org/pulp/wiki/Authentication + +[auditing] +events_file: /var/log/pulp/events.log +lifetime: 90 +backups: 4 + +# Configuration for storage of consumer history +# +# lifetime: length in days - consumer history entries older than this +# will be purged; setting this to -1 will disable history purging +[consumer_history] +lifetime: 180 + +[logs] +qpid_log_level: info +level: info +pulp_file: /var/log/pulp/pulp.log +grinder_file: /var/log/pulp/grinder.log + +[rhn] +# using less than 5 threads can cause issues with progress feed back +threads: 5 +fetch_all_packages: false +remove_old_packages: false +cert_file: /etc/sysconfig/rhn/entitlement-cert.xml +systemid_file: /etc/sysconfig/rhn/systemid + +[yum] +threads: 3 +# True/False to flag if we should remove older rpms +remove_old_packages: false +# Integer to specify how many old packages to keep. +# Only used if 'removeold' is set to True +num_old_pkgs_keep: 2 +# Uncomment below to set a bandwidth limit in KB/sec for downloading +# Only used if 'removeold' is set to True +num_old_pkgs_keep: 2 +# Uncomment below to set a bandwidth limit in KB/sec for downloading +# 0 represents unlimited +# limit_in_KB: 500 +# Uncomment the below section with appropriate values for proxy configuration +# proxy_url: +# proxy_port: +# proxy_user: +# proxy_pass: + +[repos] +content_url: https://cdn.redhat.com/ +use_entitlement_certs = false +default_to_published: true + +[database] +# automatically upgrade the database when the data model changes +name: pulp_database +# comma separated list of URIs to pass to pymongo +seeds: localhost + +# Uncomment the below section with appropriate values, to use an external ldap for +# user authentication instead of pulp. +#[ldap] +#uri: ldap://localhost +#base: dc=localhost + +[cds] +# messaging timeout for sync. <initial>:<duration> +sync_timeout = 10:7200 diff --git a/recipes/apache/manifests/init.pp b/recipes/apache/manifests/init.pp index 79f15f9..de703aa 100644 --- a/recipes/apache/manifests/init.pp +++ b/recipes/apache/manifests/init.pp @@ -5,14 +5,16 @@ class apache { # require apache and mod_ssl package { "httpd": ensure => installed } - if $enable_security { + if $enable_https { package { "mod_ssl": ensure => installed } } # if selinux is enabled and we want to use mod_proxy, we need todo this exec{'permit-http-networking': command => '/usr/sbin/setsebool httpd_can_network_connect 1', - logoutput => true } + logoutput => true, + unless => "/usr/bin/test 'Disabled' = `/usr/sbin/getenforce`" + } service { "httpd": ensure => running, @@ -38,3 +40,4 @@ define apache::site ( $ensure = 'present', $source = '') { require => Service['httpd'] } } + -- 1.7.4.4

1 0

[PATCH configure] puppet pulp sync script v2
by Richard Su 26 Apr '11

26 Apr '11

1. check pulp rpm is installed 2. write out templatized configuration files 3. start pulp-server service 4. initializes pulp 5. saves default credentials (may need to change admin password) 6. creates x86_64 repos for f13 and f14 7. triggers a sync on both repos There is an option to force the sync and wait until the sync completes before returning status to puppet. For me, the initial sync took 1hr and subsequent syncs took 2 minutes. I decided not to use the force option and let sync happen in the background. --- recipes/aeolus_recipe/aeolus_recipe.pp | 1 + recipes/aeolus_recipe/aeolus_uninstall.pp | 1 + recipes/aeolus_recipe/manifests/aeolus.pp | 2 +- recipes/aeolus_recipe/manifests/pulp.pp | 69 +++++++++++++ recipes/aeolus_recipe/templates/pulp-client.conf | 49 +++++++++ recipes/aeolus_recipe/templates/pulp.conf | 119 ++++++++++++++++++++++ 6 files changed, 240 insertions(+), 1 deletions(-) create mode 100644 recipes/aeolus_recipe/manifests/pulp.pp create mode 100755 recipes/aeolus_recipe/templates/pulp-client.conf create mode 100644 recipes/aeolus_recipe/templates/pulp.conf diff --git a/recipes/aeolus_recipe/aeolus_recipe.pp b/recipes/aeolus_recipe/aeolus_recipe.pp index dd05dc3..744d773 100644 --- a/recipes/aeolus_recipe/aeolus_recipe.pp +++ b/recipes/aeolus_recipe/aeolus_recipe.pp @@ -30,6 +30,7 @@ import "aeolus_recipe/defaults" include aeolus::conductor include aeolus::image-factory include aeolus::iwhd +include aeolus::pulp aeolus::create_bucket{"aeolus":} diff --git a/recipes/aeolus_recipe/aeolus_uninstall.pp b/recipes/aeolus_recipe/aeolus_uninstall.pp index c0b0eae..7c2d4c0 100644 --- a/recipes/aeolus_recipe/aeolus_uninstall.pp +++ b/recipes/aeolus_recipe/aeolus_uninstall.pp @@ -30,6 +30,7 @@ import "aeolus_recipe/defaults" include aeolus::conductor::disabled include aeolus::iwhd::disabled include aeolus::image-factory::disabled +include aeolus::pulp::disabled aeolus::deltacloud::disabled{"mock": } aeolus::deltacloud::disabled{"ec2-us-east-1": } diff --git a/recipes/aeolus_recipe/manifests/aeolus.pp b/recipes/aeolus_recipe/manifests/aeolus.pp index de43f9c..d94ed38 100644 --- a/recipes/aeolus_recipe/manifests/aeolus.pp +++ b/recipes/aeolus_recipe/manifests/aeolus.pp @@ -10,6 +10,7 @@ import "conductor" import "deltacloud" import "iwhd" import "image-factory" +import "pulp" if $aeolus_enable_https == "true" or $aeolus_enable_https == "1" { import "openssl" @@ -63,4 +64,3 @@ define aeolus::provider($type, $port, $login_user="", $login_password=""){ login_password => $login_password, require => Aeolus::Deltacloud[$name] } } - diff --git a/recipes/aeolus_recipe/manifests/pulp.pp b/recipes/aeolus_recipe/manifests/pulp.pp new file mode 100644 index 0000000..0c5a198 --- /dev/null +++ b/recipes/aeolus_recipe/manifests/pulp.pp @@ -0,0 +1,69 @@ +# Aeolus pulp puppet definitions + +class aeolus::pulp inherits aeolus { + package { 'pulp': + ensure => 'installed' + } + + file { 'pulp-conf-file-pulp': + path => "/etc/pulp/pulp.conf", + owner => root, + group => root, + content => template("aeolus_recipe/pulp.conf") + } + + file { 'pulp-conf-file-client': + path => "/etc/pulp/client.conf", + owner => apache, + group => apache, + content => template("aeolus_recipe/pulp-client.conf") + } + + exec{'pulp-init': + command => "/sbin/service pulp-server init", + logoutput => true, + require => [Package['pulp'], + File['pulp-conf-file-pulp'], + File['pulp-conf-file-client']]} + + service { 'pulp-server': + ensure => 'running', + enable => true, + require => [Package['pulp'], + Exec["pulp-init"]]} + + exec{'pulp-auth': + command => "/usr/bin/pulp-admin -u admin -p admin auth login --username admin --password admin", + logoutput => true, + require => Service['pulp-server']} + + exec{'pulp-f13-x86-64-repo-create': + command => "/usr/bin/pulp-admin repo create --id f13-x86_64 --feed yum:http://download.fedoraproject.org/pub/fedora/linux/releases/13/Fedora/x…", + logoutput => true, + creates => "/var/lib/pulp/repos/pub/fedora/linux/releases/13/Fedora/x86_64/os", + require => Exec['pulp-auth']} + + exec{'pulp-f13-x86-64-repo-sync': + command => "/usr/bin/pulp-admin repo sync --id f13-x86_64", + logoutput => true, + require => Exec['pulp-f13-x86-64-repo-create']} + + exec{'pulp-f14-x86-64-repo-create': + command => "/usr/bin/pulp-admin repo create --id f14-x86_64 --feed yum:http://download.fedoraproject.org/pub/fedora/linux/releases/14/Fedora/x…", + logoutput => true, + creates => "/var/lib/pulp/repos/pub/fedora/linux/releases/14/Fedora/x86_64/os", + require => Exec['pulp-auth']} + + exec{'pulp-f14-x86-64-repo-sync': + command => "/usr/bin/pulp-admin repo sync --id f14-x86_64", + logoutput => true, + require => Exec['pulp-f14-x86-64-repo-create']} +} + +class aeolus::pulp::disabled { + service { 'pulp-server': + ensure => 'stopped', + enable => false, + hasstatus=> true + } +} diff --git a/recipes/aeolus_recipe/templates/pulp-client.conf b/recipes/aeolus_recipe/templates/pulp-client.conf new file mode 100755 index 0000000..bda63d1 --- /dev/null +++ b/recipes/aeolus_recipe/templates/pulp-client.conf @@ -0,0 +1,49 @@ +# The pulp server configuration +# host : The pulp server +# port : The port providing the RESTful API. +# scheme : The protocol. +# interval : The agent update interval +[server] +host = <%= fqdn %> +port = 443 +scheme = https +path = /pulp/api +interval = 240 + +# Client configuration +# reboot_schedule : Time in minutes or 'now' before system is scheduled for reboot +# when user requests a reboot for applicable errata. +# assumeyes : Uncomment this to override this value. Assumes that install performs all the suggested actions such as reboot on successful install; +# import_gpg_keys : automatically import gpg keys if available during remote package installs +[client] +reboot_schedule = 3 +import_gpg_keys = False +#assumeyes = False +repo_file = /etc/yum.repos.d/pulp.repo +mirror_list_dir = /etc/yum.repos.d +gpg_keys_dir = /etc/pki/pulp-gpg-keys + +# heartbeat +# interval in seconds +[heartbeat] +seconds=10 + +# messaging +# scheme: (tcp|ssl) +# port: broker port +# cacert: the CA certificate (PEM) to verify the server. +# clientcert: The client (PEM) key & certificate. +[messaging] +scheme = tcp +port = 5672 +cacert = +clientcert = + + +# Content distribution options +# baseurl : The content base url +# keyurl : The gpg key base url +[cds] +baseurl = https://localhost/pulp/repos +keyurl = http://localhost/pulp/gpg +ksurl = http://localhost/pulp/ks diff --git a/recipes/aeolus_recipe/templates/pulp.conf b/recipes/aeolus_recipe/templates/pulp.conf new file mode 100644 index 0000000..e78bbf9 --- /dev/null +++ b/recipes/aeolus_recipe/templates/pulp.conf @@ -0,0 +1,119 @@ +# Server options +# +# server_name: server hostname(protocol defaults to https) +# relative_url: the relative url at which repos are exposed to the clients +# key_url: the relative url at which gpg keys are exposed to clients +# default_login: default admin username for your pulp server +# default_password: default password for admin +# Highly recommend changing the default_password with "pulp-admin user update" +# +[server] +server_name: <%= fqdn %> +relative_url: /pulp/repos +key_url: /pulp/gpg +default_login: admin +default_password: admin + +# Messaging options. +# +# url: the broker url. +# cacert: path to PEM encoded CA certificate file +# clientcert: path to PEM encoded file containing both +# the private key and certificate. +# +# The url has form: <transport>://<host>:<port> where +# transport can be (tcp|ssl). When ssl is specified, +# cacert and clientcert must be specified. + +[messaging] +url: tcp://localhost:5672 +cacert: /etc/pki/qpid/ca/ca.crt +clientcert: /etc/pki/qpid/client/client.pem + +# +# AMQP event processing +# +[events] +send_enabled: false +recv_enabled: false + +# Configures aspects of the pulp web server security. +# +# cacert: full path to the CA certificate that will be used to sign +# consumer and admin identification certificates. This MUST match +# the value of SSLCACertificateFile in /etc/httpd/conf.d/pulp.conf. +# cakey: full path to the private key for the CA certificate +# + +[security] +cacert: /etc/pki/pulp/ca.crt +cakey: /etc/pki/pulp/ca.key +# oauth_key: string key to enable OAuth style authentication +# oauth_secret: string shared secret that can be used for OAuth style authentication +# For more info see: https://fedorahosted.org/pulp/wiki/Authentication + +[auditing] +events_file: /var/log/pulp/events.log +lifetime: 90 +backups: 4 + +# Configuration for storage of consumer history +# +# lifetime: length in days - consumer history entries older than this +# will be purged; setting this to -1 will disable history purging +[consumer_history] +lifetime: 180 + +[logs] +qpid_log_level: info +level: info +pulp_file: /var/log/pulp/pulp.log +grinder_file: /var/log/pulp/grinder.log + +[rhn] +# using less than 5 threads can cause issues with progress feed back +threads: 5 +fetch_all_packages: false +remove_old_packages: false +cert_file: /etc/sysconfig/rhn/entitlement-cert.xml +systemid_file: /etc/sysconfig/rhn/systemid + +[yum] +threads: 3 +# True/False to flag if we should remove older rpms +remove_old_packages: false +# Integer to specify how many old packages to keep. +# Only used if 'removeold' is set to True +num_old_pkgs_keep: 2 +# Uncomment below to set a bandwidth limit in KB/sec for downloading +# Only used if 'removeold' is set to True +num_old_pkgs_keep: 2 +# Uncomment below to set a bandwidth limit in KB/sec for downloading +# 0 represents unlimited +# limit_in_KB: 500 +# Uncomment the below section with appropriate values for proxy configuration +# proxy_url: +# proxy_port: +# proxy_user: +# proxy_pass: + +[repos] +content_url: https://cdn.redhat.com/ +use_entitlement_certs = false +default_to_published: true + +[database] +# automatically upgrade the database when the data model changes +name: pulp_database +# comma separated list of URIs to pass to pymongo +seeds: localhost + +# Uncomment the below section with appropriate values, to use an external ldap for +# user authentication instead of pulp. +#[ldap] +#uri: ldap://localhost +#base: dc=localhost + +[cds] +# messaging timeout for sync. <initial>:<duration> +sync_timeout = 10:7200 -- 1.7.4.4

1 0

[PATCH conductor] Make ProviderAccount.build_credentials dynamic
by Tomas Sedovic 26 Apr '11

26 Apr '11

From: Tomas Sedovic <tsedovic(a)redhat.com> This fixes Redmine #692 We're building the credentials XML that's passed to Image Factory dynamically using the credential_definitions table now. The advantage is that as we add new providers, we won't have to update the build_credentials function. We populate the credential_definitions with the new entries and that will be it. Fix according to jay's suggestions Fix the tests --- src/app/models/provider_account.rb | 44 +++++++++++-------- src/app/models/push_job.rb | 2 +- ...19142700_change_credential_definitions_names.rb | 30 +++++++++++++ src/db/seeds.rb | 10 ++-- src/spec/models/provider_account_spec.rb | 4 +- 5 files changed, 63 insertions(+), 27 deletions(-) create mode 100644 src/db/migrate/20110419142700_change_credential_definitions_names.rb diff --git a/src/app/models/provider_account.rb b/src/app/models/provider_account.rb index 9858219..b1c1bb8 100644 --- a/src/app/models/provider_account.rb +++ b/src/app/models/provider_account.rb @@ -170,25 +170,31 @@ class ProviderAccount < ActiveRecord::Base end def build_credentials - xml = Nokogiri::XML <<EOT -<?xml version="1.0"?> -<provider_credentials> - <ec2_credentials> - <account_number></account_number> - <access_key></access_key> - <secret_access_key></secret_access_key> - <certificate></certificate> - <key></key> - </ec2_credentials> -</provider_credentials> -EOT - node = xml.at_xpath('/provider_credentials/ec2_credentials') - node.at_xpath('./account_number').content = credentials_hash['account_id'] - node.at_xpath('./access_key').content = credentials_hash['username'] - node.at_xpath('./secret_access_key').content = credentials_hash['password'] - node.at_xpath('./certificate').content = credentials_hash['x509public'] - node.at_xpath('./key').content = credentials_hash['x509private'] - xml + doc = Nokogiri::XML('') + doc.root = Nokogiri::XML::Node.new('provider_credentials', doc) + root = doc.root.at_xpath('/provider_credentials') + + credential_node_name = provider.provider_type.codename + '_credentials' + credential_node = Nokogiri::XML::Node.new(credential_node_name, doc) + root << credential_node + + creds_label_hash.each do |h| + element = Nokogiri::XML::Node.new(h[:label], doc) + element.content = h[:value] + credential_node << element + end + doc.to_xml + end + + def creds_label_hash + label_value_pairs = credentials.map do |c| + { :label => c.credential_definition.label.downcase.split.join('_'), + :value => c.value } + end + + # The list is ordered by labels. That way we guarantee that the resulting + # XML is always the same which makes it easier to verify in tests. + label_value_pairs.sort { |a, b| a[:label] <=> b[:label] } end def generate_auth_key diff --git a/src/app/models/push_job.rb b/src/app/models/push_job.rb index 9983953..e028b84 100644 --- a/src/app/models/push_job.rb +++ b/src/app/models/push_job.rb @@ -28,7 +28,7 @@ class PushJob < Struct.new(:provider_image_id, :hydra) # TODO: what if a provider has multiple accounts # for now pick first account provider_account = provider_image.provider.provider_accounts.first - cred_block = provider_account.build_credentials.to_xml.html_safe + cred_block = provider_account.build_credentials.html_safe response = RestClient.post(YAML.load_file("#{RAILS_ROOT}/config/image_factory_console.yml")['pushurl'], :image_id => provider_image.image.uuid, :provider => provider_image.provider.name, :credentials => cred_block diff --git a/src/db/migrate/20110419142700_change_credential_definitions_names.rb b/src/db/migrate/20110419142700_change_credential_definitions_names.rb new file mode 100644 index 0000000..a237e9e --- /dev/null +++ b/src/db/migrate/20110419142700_change_credential_definitions_names.rb @@ -0,0 +1,30 @@ +class ChangeCredentialDefinitionsNames < ActiveRecord::Migration + def self.up + CredentialDefinition.all.each do |cred| + if name_mapping.has_key? cred.label + cred.label = name_mapping[cred.label] + cred.save! + end + end + end + + def self.down + reverse_mapping = name_mapping.invert + CredentialDefinition.all.each do |cred| + if reverse_mapping.has_key? cred.label + cred.label = reverse_mapping[cred.label] + cred.save! + end + end + end + + def self.name_mapping + { + "API Key" => "Access Key", + "Secret" => "Secret Access Key", + "AWS Account ID" => "Account Number", + "EC2 x509 private key" => "Key", + "EC2 x509 public key" => "Certificate", + } + end +end diff --git a/src/db/seeds.rb b/src/db/seeds.rb index 69be3f4..363a08f 100644 --- a/src/db/seeds.rb +++ b/src/db/seeds.rb @@ -133,9 +133,9 @@ if CredentialDefinition.all.empty? #for ec2 provider type ec2 = ProviderType.find_by_codename 'ec2' - CredentialDefinition.create!(:name => 'username', :label => 'API Key', :input_type => 'text', :provider_type_id => ec2.id) - CredentialDefinition.create!(:name => 'password', :label => 'Secret', :input_type => 'password', :provider_type_id => ec2.id) - CredentialDefinition.create!(:name => 'account_id', :label => 'AWS Account ID', :input_type => 'text', :provider_type_id => ec2.id) - CredentialDefinition.create!(:name => 'x509private', :label => 'EC2 x509 private key', :input_type => 'file', :provider_type_id => ec2.id) - CredentialDefinition.create!(:name => 'x509public', :label => 'EC2 x509 public key', :input_type => 'file', :provider_type_id => ec2.id) + CredentialDefinition.create!(:name => 'username', :label => 'Access Key', :input_type => 'text', :provider_type_id => ec2.id) + CredentialDefinition.create!(:name => 'password', :label => 'Secret Access Key', :input_type => 'password', :provider_type_id => ec2.id) + CredentialDefinition.create!(:name => 'account_id', :label => 'Account Number', :input_type => 'text', :provider_type_id => ec2.id) + CredentialDefinition.create!(:name => 'x509private', :label => 'Key', :input_type => 'file', :provider_type_id => ec2.id) + CredentialDefinition.create!(:name => 'x509public', :label => 'Certificate', :input_type => 'file', :provider_type_id => ec2.id) end diff --git a/src/spec/models/provider_account_spec.rb b/src/spec/models/provider_account_spec.rb index df49708..2c3c090 100644 --- a/src/spec/models/provider_account_spec.rb +++ b/src/spec/models/provider_account_spec.rb @@ -68,11 +68,11 @@ describe ProviderAccount do <?xml version="1.0"?> <provider_credentials> <ec2_credentials> - <account_number>1234</account_number> <access_key>user</access_key> - <secret_access_key>pass</secret_access_key> + <account_number>1234</account_number> <certificate>cert</certificate> <key>priv_key</key> + <secret_access_key>pass</secret_access_key> </ec2_credentials> </provider_credentials> EOT -- 1.7.4.4

2 3

Move the Image Factory rpm location?
by Justin Clift 26 Apr '11

26 Apr '11

Hi guys, With Image Factory, when it compiles the .rpm using the spec file, the result is placed into a "dist" subdirectory. Would it be possible to have that built into ~/rpmbuild/RPMS/ ? It's not a big deal, it's just a consistency thing. All of the other projects build into the ~/rpmbuild/ structure, and one location is easier to write instructions for. :) Regards and best wishes, Justin Clift -- Aeolus Community Manager http://www.aeolusproject.org

2 1

task #1112 - aeolus-configure documentation
by Mo Morsi 26 Apr '11

26 Apr '11

3 3

[PATCH configure] puppet pulp sync script
by Richard Su 25 Apr '11

25 Apr '11

1. check pulp rpm is installed 2. start pulp-server service 3. initializes pulp 4. saves default credentials (may need to change admin password) 5. creates x86_64 repos for f13 and f14 6. triggers a sync on both repos There is an option to force the sync and wait until the sync completes before returning status to puppet. For me, the initial sync took 1hr and subsequent syncs took 2 minutes. I decided not to use the force option and let sync happen in the background. --- recipes/aeolus_recipe/aeolus_recipe.pp | 1 + recipes/aeolus_recipe/aeolus_uninstall.pp | 1 + recipes/aeolus_recipe/manifests/aeolus.pp | 2 +- recipes/aeolus_recipe/manifests/pulp.pp | 53 +++++++++++++++++++++++++++++ 4 files changed, 56 insertions(+), 1 deletions(-) create mode 100644 recipes/aeolus_recipe/manifests/pulp.pp diff --git a/recipes/aeolus_recipe/aeolus_recipe.pp b/recipes/aeolus_recipe/aeolus_recipe.pp index dd05dc3..744d773 100644 --- a/recipes/aeolus_recipe/aeolus_recipe.pp +++ b/recipes/aeolus_recipe/aeolus_recipe.pp @@ -30,6 +30,7 @@ import "aeolus_recipe/defaults" include aeolus::conductor include aeolus::image-factory include aeolus::iwhd +include aeolus::pulp aeolus::create_bucket{"aeolus":} diff --git a/recipes/aeolus_recipe/aeolus_uninstall.pp b/recipes/aeolus_recipe/aeolus_uninstall.pp index c0b0eae..7c2d4c0 100644 --- a/recipes/aeolus_recipe/aeolus_uninstall.pp +++ b/recipes/aeolus_recipe/aeolus_uninstall.pp @@ -30,6 +30,7 @@ import "aeolus_recipe/defaults" include aeolus::conductor::disabled include aeolus::iwhd::disabled include aeolus::image-factory::disabled +include aeolus::pulp::disabled aeolus::deltacloud::disabled{"mock": } aeolus::deltacloud::disabled{"ec2-us-east-1": } diff --git a/recipes/aeolus_recipe/manifests/aeolus.pp b/recipes/aeolus_recipe/manifests/aeolus.pp index de43f9c..d94ed38 100644 --- a/recipes/aeolus_recipe/manifests/aeolus.pp +++ b/recipes/aeolus_recipe/manifests/aeolus.pp @@ -10,6 +10,7 @@ import "conductor" import "deltacloud" import "iwhd" import "image-factory" +import "pulp" if $aeolus_enable_https == "true" or $aeolus_enable_https == "1" { import "openssl" @@ -63,4 +64,3 @@ define aeolus::provider($type, $port, $login_user="", $login_password=""){ login_password => $login_password, require => Aeolus::Deltacloud[$name] } } - diff --git a/recipes/aeolus_recipe/manifests/pulp.pp b/recipes/aeolus_recipe/manifests/pulp.pp new file mode 100644 index 0000000..cf97827 --- /dev/null +++ b/recipes/aeolus_recipe/manifests/pulp.pp @@ -0,0 +1,53 @@ +# Aeolus pulp puppet definitions + +class aeolus::pulp inherits aeolus { + package { 'pulp': + ensure => 'installed' + } + + exec{'pulp-init': + command => "/sbin/service pulp-server init", + logoutput => true, + require => Package['pulp']} + + service { 'pulp-server': + ensure => 'running', + enable => true, + require => [Package['pulp'], + Exec["pulp-init"]]} + + exec{'pulp-auth': + command => "/usr/bin/pulp-admin -u admin -p admin auth login --username admin --password admin", + logoutput => true, + require => Service['pulp-server']} + + exec{'pulp-f13-x86-64-repo-create': + command => "/usr/bin/pulp-admin repo create --id f13-x86_64 --feed yum:http://download.fedoraproject.org/pub/fedora/linux/releases/13/Fedora/x…", + logoutput => true, + creates => "/var/lib/pulp/repos/pub/fedora/linux/releases/13/Fedora/x86_64/os", + require => Exec['pulp-auth']} + + exec{'pulp-f13-x86-64-repo-sync': + command => "/usr/bin/pulp-admin repo sync --id f13-x86_64", + logoutput => true, + require => Exec['pulp-f13-x86-64-repo-create']} + + exec{'pulp-f14-x86-64-repo-create': + command => "/usr/bin/pulp-admin repo create --id f14-x86_64 --feed yum:http://download.fedoraproject.org/pub/fedora/linux/releases/14/Fedora/x…", + logoutput => true, + creates => "/var/lib/pulp/repos/pub/fedora/linux/releases/14/Fedora/x86_64/os", + require => Exec['pulp-auth']} + + exec{'pulp-f14-x86-64-repo-sync': + command => "/usr/bin/pulp-admin repo sync --id f14-x86_64", + logoutput => true, + require => Exec['pulp-f14-x86-64-repo-create']} +} + +class aeolus::pulp::disabled { + service { 'pulp-server': + ensure => 'stopped', + enable => false, + hasstatus=> true + } +} -- 1.7.4.4

1 0

#694060 - Handle image import errors
by Matt Wagner 25 Apr '11

25 Apr '11

This patch fixes #694060, in which attempting to import an EC2 AMI that didn't exist raised an exception that was not handled. The existing code was not catching the exception properly, and the exception it was trying to show contained multiple lines of debugging info. This should clean it up. Unfortunately, the test I added isn't able to test the exact failure (it uses mock, not ec2), but it should still apply. -- Matt

2 2

[PATCH configure] BZ #693369: ensure deltacloud-core is actually running before returning from init script
by Mo Morsi 25 Apr '11

25 Apr '11

--- recipes/aeolus_recipe/templates/deltacloud-core | 46 +++++++++++++++------- 1 files changed, 31 insertions(+), 15 deletions(-) diff --git a/recipes/aeolus_recipe/templates/deltacloud-core b/recipes/aeolus_recipe/templates/deltacloud-core index d5cd141..8cc998a 100644 --- a/recipes/aeolus_recipe/templates/deltacloud-core +++ b/recipes/aeolus_recipe/templates/deltacloud-core @@ -20,26 +20,42 @@ export DELTACLOUD_MOCK_STORAGE=/usr/lib/ruby/gems/1.8/gems/deltacloud-<%= name % PROG=/usr/bin/deltacloudd +STARTTIMEOUT=20 + . /etc/init.d/functions start() { echo -n "Starting deltacloud-<%= name %>: " - if test -f $LOCKFILE ; then - echo_success - echo - else - $PROG -i $DRIVER -e $ENV -p $PORT >> $LOGFILE 2>&1 & - RETVAL=$? - echo $! > $PIDFILE - if [ $RETVAL -eq 0 ] && touch $LOCKFILE ; then - echo_success - echo - else - echo_failure - echo - fi - fi + if [ -f $LOCKFILE ] || [ -f $PIDFILE ] && checkpid `cat $PIDFILE` ; then + echo_success + echo + echo "deltacloud-$DRIVER has already been started" + exit 0 + fi + + $PROG -i $DRIVER -e $ENV -p $PORT >> $LOGFILE 2>&1 & + RETVAL=$? + echo $! > $PIDFILE + if [ $RETVAL -eq 0 ] ; then + TIMEOUT="$STARTTIMEOUT" + while [ $TIMEOUT -gt 0 ]; do + /usr/bin/curl --silent http://localhost:$PORT/api >& /dev/null + RETVAL=$? + if [ $RETVAL -eq 0 ] ; then + touch $LOCKFILE + echo_success + echo + exit 0 + fi + sleep 1 + let TIMEOUT=${TIMEOUT}-1 + done + fi + + echo_failure + echo + exit 1 } stop() { -- 1.7.2.3

2 1

[PATCH configure] story 1241: Allow https to be toggled indendently of other security features
by Mike Orazi 25 Apr '11

25 Apr '11

This patchset allows https to be toggled indepedently of other security configurations. Additionally, the curl invocation used to seed data has been updated to correctly follow redirects so it works whether the http or https setup is selected.

2 3

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

aeolus-devel April 2011