From: Michael Scherer misc@zarb.org
--- roles/mailman/files/memcached.sysconfig | 5 ----- roles/mailman/handlers/main.yml | 3 +-- roles/mailman/meta/main.yml | 4 ++++ roles/mailman/tasks/main.yml | 10 ---------- roles/memcached/files/memcached.sysconfig | 5 +++++ roles/memcached/handlers/main.yml | 3 +++ roles/memcached/tasks/main.yml | 15 +++++++++++++++ 7 files changed, 28 insertions(+), 17 deletions(-) delete mode 100644 roles/mailman/files/memcached.sysconfig create mode 100644 roles/mailman/meta/main.yml create mode 100644 roles/memcached/files/memcached.sysconfig create mode 100644 roles/memcached/handlers/main.yml create mode 100644 roles/memcached/tasks/main.yml
diff --git a/roles/mailman/files/memcached.sysconfig b/roles/mailman/files/memcached.sysconfig deleted file mode 100644 index 6a53e3e..0000000 --- a/roles/mailman/files/memcached.sysconfig +++ /dev/null @@ -1,5 +0,0 @@ -PORT="11211" -USER="memcached" -MAXCONN="1024" -CACHESIZE="256" -OPTIONS="" diff --git a/roles/mailman/handlers/main.yml b/roles/mailman/handlers/main.yml index ee96b7a..0761d2f 100644 --- a/roles/mailman/handlers/main.yml +++ b/roles/mailman/handlers/main.yml @@ -8,5 +8,4 @@ - name: reload apache action: service name=httpd state=reloaded
-- name: restart memcached - action: service name=memcached state=restarted + diff --git a/roles/mailman/meta/main.yml b/roles/mailman/meta/main.yml new file mode 100644 index 0000000..ccebbea --- /dev/null +++ b/roles/mailman/meta/main.yml @@ -0,0 +1,4 @@ +--- +dependencies: +- { role: memcached } + diff --git a/roles/mailman/tasks/main.yml b/roles/mailman/tasks/main.yml index 686f3db..a4b338e 100644 --- a/roles/mailman/tasks/main.yml +++ b/roles/mailman/tasks/main.yml @@ -45,7 +45,6 @@ - hyperkitty - hyperkitty-selinux - postorius - - memcached - python-pylibmc - yum-plugin-post-transaction-actions - mailman3-fedmsg-plugin @@ -235,12 +234,3 @@ - name: create the postfix aliases command: su mailman -s /bin/sh -c "mailman3 aliases" creates=/var/lib/mailman3/data/postfix_lmtp.db - -# Memcached -- name: set the memcached sysconfig file - copy: src=memcached.sysconfig dest=/etc/sysconfig/memcached - notify: - - restart memcached - -- name: start memcached - service: state=started enabled=yes name=memcached diff --git a/roles/memcached/files/memcached.sysconfig b/roles/memcached/files/memcached.sysconfig new file mode 100644 index 0000000..6a53e3e --- /dev/null +++ b/roles/memcached/files/memcached.sysconfig @@ -0,0 +1,5 @@ +PORT="11211" +USER="memcached" +MAXCONN="1024" +CACHESIZE="256" +OPTIONS="" diff --git a/roles/memcached/handlers/main.yml b/roles/memcached/handlers/main.yml new file mode 100644 index 0000000..5af2e59 --- /dev/null +++ b/roles/memcached/handlers/main.yml @@ -0,0 +1,3 @@ +--- +- name: restart memcached + action: service name=memcached state=restarted diff --git a/roles/memcached/tasks/main.yml b/roles/memcached/tasks/main.yml new file mode 100644 index 0000000..cd591bc --- /dev/null +++ b/roles/memcached/tasks/main.yml @@ -0,0 +1,15 @@ +--- +- name: install needed packages + yum: pkg={{ item }} state=installed + with_items: + - memcached + tags: + - packages + +- name: set the memcached sysconfig file + copy: src=memcached.sysconfig dest=/etc/sysconfig/memcached + notify: + - restart memcached + +- name: start memcached + service: state=started enabled=yes name=memcached
From: Michael Scherer misc@zarb.org
--- roles/badges/frontend/meta/main.yml | 3 +++ roles/mailman/meta/main.yml | 1 - roles/nuancier/meta/main.yml | 4 ++++ roles/pkgdb2/meta/main.yml | 3 +++ 4 files changed, 10 insertions(+), 1 deletion(-) create mode 100644 roles/badges/frontend/meta/main.yml create mode 100644 roles/nuancier/meta/main.yml create mode 100644 roles/pkgdb2/meta/main.yml
diff --git a/roles/badges/frontend/meta/main.yml b/roles/badges/frontend/meta/main.yml new file mode 100644 index 0000000..685b148 --- /dev/null +++ b/roles/badges/frontend/meta/main.yml @@ -0,0 +1,3 @@ +--- +dependencies: +- { role: memcached } diff --git a/roles/mailman/meta/main.yml b/roles/mailman/meta/main.yml index ccebbea..685b148 100644 --- a/roles/mailman/meta/main.yml +++ b/roles/mailman/meta/main.yml @@ -1,4 +1,3 @@ --- dependencies: - { role: memcached } - diff --git a/roles/nuancier/meta/main.yml b/roles/nuancier/meta/main.yml new file mode 100644 index 0000000..ccebbea --- /dev/null +++ b/roles/nuancier/meta/main.yml @@ -0,0 +1,4 @@ +--- +dependencies: +- { role: memcached } + diff --git a/roles/pkgdb2/meta/main.yml b/roles/pkgdb2/meta/main.yml new file mode 100644 index 0000000..685b148 --- /dev/null +++ b/roles/pkgdb2/meta/main.yml @@ -0,0 +1,3 @@ +--- +dependencies: +- { role: memcached }
The first patch here looks fine to me... but the second one isn't going to work I don't think.
Those services use memcached, but they use our existing memcached03/04 instances that are still in puppet, not local memcached.
I guess we should think about how we want to use memcached moving forward some. Do we want dedicated instances? Or do we want to prefer using local caches on each applications servers? Using dedicated servers means we could share more, but local might still be good enough...
thoughts?
kevin
On Thu, Apr 17, 2014 at 10:19:02AM -0600, Kevin Fenzi wrote:
The first patch here looks fine to me... but the second one isn't going to work I don't think.
Those services use memcached, but they use our existing memcached03/04 instances that are still in puppet, not local memcached.
Not all. Nuancier seems to use localhost : http://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/nuancier...
So does pkgdb : http://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/pkgdb2/t...
( unless i missed something and pkgdb2 and nuancier run on a memcache server )
But indeed, badges do use memcached03, I wonder how I missed that :/
I guess we should think about how we want to use memcached moving forward some. Do we want dedicated instances? Or do we want to prefer using local caches on each applications servers? Using dedicated servers means we could share more, but local might still be good enough...
Shared servers would help if more than 1 server use it, like 2 redundant applications.
But having local memcache permit to know how much memory each service use, and to restart them selectively. It would also reduce the network trafic and likely improve latency. ( depending on the topology, I didn't really look at it )
On Thu, Apr 17, 2014 at 08:48:00PM +0200, Michael scherer wrote:
On Thu, Apr 17, 2014 at 10:19:02AM -0600, Kevin Fenzi wrote:
The first patch here looks fine to me... but the second one isn't going to work I don't think.
Those services use memcached, but they use our existing memcached03/04 instances that are still in puppet, not local memcached.
Not all. Nuancier seems to use localhost : http://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/nuancier...
So does pkgdb : http://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/pkgdb2/t...
( unless i missed something and pkgdb2 and nuancier run on a memcache server )
Nuancier and pkgdb2 could be moved, it's just a matter of adjusting the configuration file. pkgdb2 has memcache but iirc, it doesn't do much with it. I tried to speed up some of the API calls with it but it wasn't really a clear win, so I'm not sure it actually uses memcache at this time.
Pierre
On Thu, Apr 17, 2014 at 10:28:37PM +0200, Pierre-Yves Chibon wrote:
On Thu, Apr 17, 2014 at 08:48:00PM +0200, Michael scherer wrote:
On Thu, Apr 17, 2014 at 10:19:02AM -0600, Kevin Fenzi wrote:
The first patch here looks fine to me... but the second one isn't going to work I don't think.
Those services use memcached, but they use our existing memcached03/04 instances that are still in puppet, not local memcached.
Not all. Nuancier seems to use localhost : http://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/nuancier...
So does pkgdb : http://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/pkgdb2/t...
( unless i missed something and pkgdb2 and nuancier run on a memcache server )
Nuancier and pkgdb2 could be moved, it's just a matter of adjusting the configuration file. pkgdb2 has memcache but iirc, it doesn't do much with it. I tried to speed up some of the API calls with it but it wasn't really a clear win, so I'm not sure it actually uses memcache at this time.
I correct myself, pkgdb2 still uses memcache for its integration with FAS.
Pierre
On Tue, 29 Apr 2014 07:28:30 +0200 Pierre-Yves Chibon pingou@pingoured.fr wrote:
On Thu, Apr 17, 2014 at 10:28:37PM +0200, Pierre-Yves Chibon wrote:
On Thu, Apr 17, 2014 at 08:48:00PM +0200, Michael scherer wrote:
On Thu, Apr 17, 2014 at 10:19:02AM -0600, Kevin Fenzi wrote:
The first patch here looks fine to me... but the second one isn't going to work I don't think.
Those services use memcached, but they use our existing memcached03/04 instances that are still in puppet, not local memcached.
Not all. Nuancier seems to use localhost : http://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/nuancier...
So does pkgdb : http://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/pkgdb2/t...
( unless i missed something and pkgdb2 and nuancier run on a memcache server )
Nuancier and pkgdb2 could be moved, it's just a matter of adjusting the configuration file. pkgdb2 has memcache but iirc, it doesn't do much with it. I tried to speed up some of the API calls with it but it wasn't really a clear win, so I'm not sure it actually uses memcache at this time.
I correct myself, pkgdb2 still uses memcache for its integration with FAS.
So, I was kind of wondering if we should have some kind of best practice around memcache or guidelines for Fedora Infrastructure applications.
a) When should you use memcache?
b) When should it be external host vs local?
c) How large should it be?
I think my wild ass non application developer answers would be:
a) When it's easy to integrate and you have things that would speed up by using it.
b) It should be external when you have more than 1 server running the same application (so they can share cache?)
c) no idea on this one. ;)
So, basically I think we should make a best practices and then update all our applications to do things that way. ;) If that means we need more seperate memcached machines, fine. If it means we need to remove all those and go with seperate caches, thats fine too. I just want us to not be doing lots of different things on different apps where there's no reason for the difference. ;)
kevin
On Tue, Apr 29, 2014 at 10:01:49AM -0600, Kevin Fenzi wrote:
On Tue, 29 Apr 2014 07:28:30 +0200 Pierre-Yves Chibon pingou@pingoured.fr wrote:
On Thu, Apr 17, 2014 at 10:28:37PM +0200, Pierre-Yves Chibon wrote:
On Thu, Apr 17, 2014 at 08:48:00PM +0200, Michael scherer wrote:
On Thu, Apr 17, 2014 at 10:19:02AM -0600, Kevin Fenzi wrote:
The first patch here looks fine to me... but the second one isn't going to work I don't think.
Those services use memcached, but they use our existing memcached03/04 instances that are still in puppet, not local memcached.
Not all. Nuancier seems to use localhost : http://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/nuancier...
So does pkgdb : http://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/pkgdb2/t...
( unless i missed something and pkgdb2 and nuancier run on a memcache server )
Nuancier and pkgdb2 could be moved, it's just a matter of adjusting the configuration file. pkgdb2 has memcache but iirc, it doesn't do much with it. I tried to speed up some of the API calls with it but it wasn't really a clear win, so I'm not sure it actually uses memcache at this time.
I correct myself, pkgdb2 still uses memcache for its integration with FAS.
So, I was kind of wondering if we should have some kind of best practice around memcache or guidelines for Fedora Infrastructure applications.
a) When should you use memcache?
b) When should it be external host vs local?
c) How large should it be?
I think my wild ass non application developer answers would be:
a) When it's easy to integrate and you have things that would speed up by using it.
b) It should be external when you have more than 1 server running the same application (so they can share cache?)
c) no idea on this one. ;)
My opinion is close to yours on most points: a) where it makes sense (speed things up, data that can spare being 59 minutes old...)
b) I guess external makes sense since we keep having 2 production instances, but I'll have to figure out how to configure it properly then
c) No idea either :)
Pierre
infrastructure@lists.fedoraproject.org
-
Kevin Fenzi -
Michael Scherer -
Pierre-Yves Chibon