The trouble with torrents

[Kevin Fenzi]

So, Fedora 29 Beta is out the door and mostly that went very smoothly as
always. With one exception: torrents.

Right now we are (were) using the last opensource version of
'bittorrent' python client to seed our torrents. However, I noticed
yesterday that it is no longer working. It's not just one bug, but a
number of them judging by the different tracebacks it's spewing.

As a stopgap to get Beta happy I installed and ran rtorrent on torrent01
in a tmux session to seed our beta torrents, but thats a curses app
thats not automateable.

So, we really need to figure a way forward here.

1. Get permission from Council/FESCo to drop torrents (To make that case
we need to show some graphs that not many people are using torrents
anymore).

2. Move our torrents to amazon cloudfront. This should be doable, but
means we are using a non free service so it would make me sad.

3. Try and find a open source seeding app that is manageable from
ansible/config files. (Good luck, they are all old, or gui only, etc).

4. Just use rtorrent in tmux to seed everything and have to manually
start/stop it when you add/remove any.

5. Some other clever idea someone has.

We can discuss this in meeting tomorrow, but I figured I would start a
thread here too.

kevin

[Marcin Skarbek]

rtorrent can be managed via XML RPC [1]. There are some python 
implementations [2] so with some effort that should allow automation of 
seeding.

1. https://github.com/rakshasa/rtorrent/wiki/RPC-Setup-XMLRPC
2. https://pypi.org/project/rtorrent-python/

Marcin

On 9/26/18 8:03 PM, Kevin Fenzi wrote:
> So, Fedora 29 Beta is out the door and mostly that went very smoothly as
> always. With one exception: torrents.
>
> Right now we are (were) using the last opensource version of
> 'bittorrent' python client to seed our torrents. However, I noticed
> yesterday that it is no longer working. It's not just one bug, but a
> number of them judging by the different tracebacks it's spewing.
>
> As a stopgap to get Beta happy I installed and ran rtorrent on torrent01
> in a tmux session to seed our beta torrents, but thats a curses app
> thats not automateable.
>
> So, we really need to figure a way forward here.
> Get permission from Council/FESCo to drop torrents (To make that case
>
> we need to show some graphs that not many people are using torrents
> anymore).
> Move our torrents to amazon cloudfront. This should be doable, but
>
> means we are using a non free service so it would make me sad.
> Try and find a open source seeding app that is manageable from
>
> ansible/config files. (Good luck, they are all old, or gui only, etc).
> Just use rtorrent in tmux to seed everything and have to manually
>
> start/stop it when you add/remove any.
> Some other clever idea someone has.
>
> We can discuss this in meeting tomorrow, but I figured I would start a
> thread here too.
>
> kevin
>
> infrastructure mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]...

[Kevin Fenzi]

On 09/26/2018 12:20 PM, Marcin Skarbek wrote:
> rtorrent can be managed via XML RPC [1]. There are some python
> implementations [2] so with some effort that should allow automation of
> seeding.
> https://github.com/rakshasa/rtorrent/wiki/RPC-Setup-XMLRPC
> https://pypi.org/project/rtorrent-python/
>
> Marcin

Nice thought. Turns out they also added a daemon mode (no ui) in the
latest release along with a way to make it use inotify to watch a dir
and just load and seed anything that gets dropped in there.

So, this might work for us... will need to investigate.

kevin

[Pablo Iranzo Gómez]

Hi Kevin,

What about using transmission-daemon which is available in Fedora?

(also console with optional Web UI)

Regards,
Pablo

----- Mensaje original -----
> De: "Kevin Fenzi" 
> Para: "Fedora Infrastructure" 
> Enviados: Miércoles, 26 de Septiembre 2018 20:03:16
> Asunto: The trouble with torrents
>
> So, Fedora 29 Beta is out the door and mostly that went very smoothly
> as
> always. With one exception: torrents.
>
> Right now we are (were) using the last opensource version of
> 'bittorrent' python client to seed our torrents. However, I noticed
> yesterday that it is no longer working. It's not just one bug, but a
> number of them judging by the different tracebacks it's spewing.
>
> As a stopgap to get Beta happy I installed and ran rtorrent on
> torrent01
> in a tmux session to seed our beta torrents, but thats a curses app
> thats not automateable.
>
> So, we really need to figure a way forward here.
> Get permission from Council/FESCo to drop torrents (To make that
>
> case
> we need to show some graphs that not many people are using torrents
> anymore).
> Move our torrents to amazon cloudfront. This should be doable, but
>
> means we are using a non free service so it would make me sad.
> Try and find a open source seeding app that is manageable from
>
> ansible/config files. (Good luck, they are all old, or gui only,
> etc).
> Just use rtorrent in tmux to seed everything and have to manually
>
> start/stop it when you add/remove any.
> Some other clever idea someone has.
>
> We can discuss this in meeting tomorrow, but I figured I would start
> a
> thread here too.
>
> kevin
>
> infrastructure mailing list -- [e-mail]
> To unsubscribe send an email to
> [e-mail]
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines:
> https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]...

[Kevin Fenzi]

On 09/26/2018 10:59 PM, Pablo Iranzo Gómez wrote:
> Hi Kevin,
>
> What about using transmission-daemon which is available in Fedora?
>
> (also console with optional Web UI)

Yeah, we looked at it the last time, but it there were issues...

I think the daemon version still pulled in the entire gtk stack or you
couldn't
manage it via config files. Looks like it might be better now...

kevin

[Mikolaj Izdebski]

On 09/29/2018 01:24 AM, Kevin Fenzi wrote:
> On 09/26/2018 10:59 PM, Pablo Iranzo Gómez wrote:
> > Hi Kevin,
> >
> > What about using transmission-daemon which is available in Fedora?
> >
> > (also console with optional Web UI)
>
> Yeah, we looked at it the last time, but it there were issues...
>
> I think the daemon version still pulled in the entire gtk stack or you
> couldn't
> manage it via config files. Looks like it might be better now...

I've been successfully using transmission-daemon since January. It can
definitely be managed via config files, but it expects to own config
files - it overwrites them when the process terminates. This can be
worked around easily, eg. by updating config files when the deamon is
not running:

- name: Create transmission config directory
  file: dest=/var/lib/transmission/.config/transmission-daemon/
state=directory owner=transmission group=transmission

- name: Copy transmission config file
  template: src=settings.json.j2
dest=/var/lib/transmission/.config/transmission-daemon/settings.json
  register: settings_j2

- name: Stop transmission-daemon
  service: name=transmission-daemon state=stopped
  when: settings_j2.changed

- name: Copy transmission config file again
  template: src=settings.json.j2
dest=/var/lib/transmission/.config/transmission-daemon/settings.json
  when: settings_j2.changed

- name: Start and enable transmission-daemon
  service: name=transmission-daemon state=started enabled=true

[David Shier]

So, I was reading the other day that aria2 also has control over 
xmlrpc, and a web ui (iirc it is no packaged with aria, but another bit 
o the project.....). So that could be an option as well. aria is my go 
to when I torrent, and has always seemed solid.

Am So, 30. Sep, 2018 um 1:27 A. M. schrieb Mikolaj Izdebski 
:
> On 09/29/2018 01:24 AM, Kevin Fenzi wrote:
> > On 09/26/2018 10:59 PM, Pablo Iranzo Gómez wrote:
> > > Hi Kevin,
> > >
> > > What about using transmission-daemon which is available in Fedora?
> > >
> > > (also console with optional Web UI)
> >
> > Yeah, we looked at it the last time, but it there were issues...
> >
> > I think the daemon version still pulled in the entire gtk stack or 
> > you
> >  couldn't
> >  manage it via config files. Looks like it might be better now...
>
> I've been successfully using transmission-daemon since January. It can
> definitely be managed via config files, but it expects to own config
> files - it overwrites them when the process terminates. This can be
> worked around easily, eg. by updating config files when the deamon is
> not running:
> name: Create transmission config directory
> file: dest=/var/lib/transmission/.config/transmission-daemon/
>
> state=directory owner=transmission group=transmission
> name: Copy transmission config file
> template: src=settings.json.j2
>
> dest=/var/lib/transmission/.config/transmission-daemon/settings.json
>   register: settings_j2
>
> name: Stop transmission-daemon
> service: name=transmission-daemon state=stopped
> when: settings_j2.changed
>
> name: Copy transmission config file again
> template: src=settings.json.j2
>
> dest=/var/lib/transmission/.config/transmission-daemon/settings.json
>   when: settings_j2.changed
> name: Start and enable transmission-daemon
> service: name=transmission-daemon state=started enabled=true
>
> --
> Mikolaj Izdebski
> Senior Software Engineer, Red Hat
> IRC: mizdebsk
> _______________________________________________
> infrastructure mailing list -- [e-mail]
> To unsubscribe send an email to 
> [e-mail]
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: 
> https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/[e-mail]...

[Cătălin George Feștilă]

In my opinion, I would remove this option with torrents from the Fedora
distribution.
It is not useful and loads with additional packets and sized.

On Wed, Sep 26, 2018 at 9:03 PM Kevin Fenzi  wrote:
> So, Fedora 29 Beta is out the door and mostly that went very smoothly as
> always. With one exception: torrents.
>
> Right now we are (were) using the last opensource version of
> 'bittorrent' python client to seed our torrents. However, I noticed
> yesterday that it is no longer working. It's not just one bug, but a
> number of them judging by the different tracebacks it's spewing.
>
> As a stopgap to get Beta happy I installed and ran rtorrent on torrent01
> in a tmux session to seed our beta torrents, but thats a curses app
> thats not automateable.
>
> So, we really need to figure a way forward here.
> Get permission from Council/FESCo to drop torrents (To make that case
>
> we need to show some graphs that not many people are using torrents
> anymore).
> Move our torrents to amazon cloudfront. This should be doable, but
>
> means we are using a non free service so it would make me sad.
> Try and find a open source seeding app that is manageable from
>
> ansible/config files. (Good luck, they are all old, or gui only, etc).
> Just use rtorrent in tmux to seed everything and have to manually
>
> start/stop it when you add/remove any.
> Some other clever idea someone has.
>
> We can discuss this in meeting tomorrow, but I figured I would start a
> thread here too.
>
> kevin
>
> infrastructure mailing list -- [e-mail]
> To unsubscribe send an email to
> [e-mail]
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]...

[Mohan Boddu]

On Wed, Sep 26, 2018 at 2:03 PM Kevin Fenzi  wrote:
> So, Fedora 29 Beta is out the door and mostly that went very smoothly as
> always. With one exception: torrents.
>
> Right now we are (were) using the last opensource version of
> 'bittorrent' python client to seed our torrents. However, I noticed
> yesterday that it is no longer working. It's not just one bug, but a
> number of them judging by the different tracebacks it's spewing.
>
> As a stopgap to get Beta happy I installed and ran rtorrent on torrent01
> in a tmux session to seed our beta torrents, but thats a curses app
> thats not automateable.
>
> So, we really need to figure a way forward here.
> Get permission from Council/FESCo to drop torrents (To make that case
>
> we need to show some graphs that not many people are using torrents
> anymore).
> Move our torrents to amazon cloudfront. This should be doable, but
>
> means we are using a non free service so it would make me sad.
> Try and find a open source seeding app that is manageable from
>
> ansible/config files. (Good luck, they are all old, or gui only, etc).

I really prefer this but 4 can be a temporary fix.
> Just use rtorrent in tmux to seed everything and have to manually
>
> start/stop it when you add/remove any.

This could be a temp fix.
> Some other clever idea someone has.
>
> We can discuss this in meeting tomorrow, but I figured I would start a
> thread here too.
>
> kevin
>
> infrastructure mailing list -- [e-mail]
> To unsubscribe send an email to
> [e-mail]
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]...