Future of Badges

[Clement Verna]

Hi all,

Today a few of us had a productive meeting to discuss the future of
the Badges application.

As a summary we have agreed on the following :

1 - No outreachy intern for this winter, but starting to prepare a
plan for next summer round.

2 - A new Front end for the application based on a JavaScript
framework and independent from the back end will be written bases on
the user trials and mock-ups. (Current plan is to have the inter work
on this next summer)

3 - Backend is hard, different possibility depending on how developer
time we have. Following are different options that need to be
discussed before moving forward.

1 - Focus on the current pain point and fix them. Hope is to improve
the current application performance
2 - Modify the back end so that it serves only JSON, consumed by the
new Front end
3 - Use the current code base and move it to Django
4 - Restart from scratch
5 - Move away from the message bus driven actions to award badges to
make the application
more generic and attract other community than Fedora.

Feedbacks are welcome, also there is Virtual Hack-fest happening
September 13-15th where it would be good to start coordinating some of
the backend work.

Thanks all

Following are the minutes:

## Attendees

Clément
Michal
Marie
Sayan
pingou


## Notes

* Frontend will be rewritten
* Backend ? is this worth the rewrite ?
    * Pyramid as a framework does not have a strong community and adoption
    * Django is the industry standard for Python based web application
* Current application is very slow
    * Badges image are big
    * Needs to download a lot of data to load a page
    * If rewrite this need to be kept in mind
    * Python2 dependencies not maintained
* Rewrite could also be porting current app to Django and slim down
the number of pages/feature
    * maybe 1 user page and 1 admin page.

* Start with the frontend --using a JS framework and let the backend
with pyramid but just serve JSON instead of templates.

* Refactoring Backend is difficult --> need experienced developer
there and a specification
    * Focusing in on the pain point first will give improvement performance

* Moving off the message driven backend would be good in order to make
the application more generic and not only Fedora centric.
    * Potential to get more contributors from other communities.

* Virtual Hackfest - September 13-15th
    * Idea to sit down and look at the technical pain point. Slow
Requests, Exception in the consumer etc
    * Write these down and come up with a plan to improve these.

[Miro Hrončok]

On 21. 08. 19 14:11, Clement Verna wrote:
>  * Backend ? is this worth the rewrite ?
>       * Pyramid as a framework does not have a strong community and adoption
>       * Django is the industry standard for Python based web application 
I'm, not arguing against Django, but I wanted to share my experience:

Apps written ion Django tend to get outdated very quickly. You need to take care 
of them if you want to support never Django versions.

OTOH Apps written in Flask tend to remain very functional for ages. I also think 
that Flask is the industry standard as well.

Please consider this aspect before choosing Django.

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Xavier トーマス LAMIEN]

>  Hi all, 
Sorry guy, I couldn't make it in time for the meeting. 
My feedback below.

<wip>
>  
>  ## Notes
>  
>  * Frontend will be rewritten
>  * Backend ? is this worth the rewrite ?
>      * Pyramid as a framework does not have a strong community and adoption
>      * Django is the industry standard for Python based web application 
I did audit the code of both API and the monolithic pyramid app after flock and came to
the same conclusion (besides some design choices which don't make sense).
However, I would say not to rewrite the backend but to merge some of the backend's
badges management/call-flow w/in the API and provide only one service.
The frontend will then relies on this API.
This will also provide to admins a better interface to manage badges (awards, update,
adding, promote, etc).

>  * Current application is very slow
>      * Badges image are big 
What I noticed is not all images are the same regarding sizing, ratio, format (looks like
more w/ the oldest). 
Don't get me wrong, I'm talking about the image object, not the badge itself

>      * Needs to download a lot of data to load a page
>      * If rewrite this need to be kept in mind 
This is a design concerns I'd say. Also, need to take into account the servers
(including their location) which serve those files.
We will have to make some architecture choices on how/what to display based on the page
the end-user is on and the load the page requires.

>      * Python2 dependencies not maintained
>  * Rewrite could also be porting current app to Django and slim down
>  the number of pages/feature
>      * maybe 1 user page and 1 admin page. 
+1 w/ Miro's comments
I'd be in favor of using AIOHTTP which provides pretty good efficiency regarding
server load/speed, good feature out-of-the-box and so much more. I'd be glad to tell
more about it if you're interested in for those who don't know.
Also, we need to think about how an IT infrastructure will deploy such services based on
their architecture. How many instances, do they need a reverse-proxy in front of them, the
daily requests load, etc?

>  
>  * Start with the frontend --using a JS framework and let the backend
>  with pyramid but just serve JSON instead of templates. 
As I said above, I'd be in favor of removing pyramid (even though I support pyramid
and was one of those who voted to have it as a selected framework for infra dev)
But we need at some point to get thing right w/ the right tools when needed and for what I
say, I don't think Pyramid, Django would be a good fit.

>  
>  * Refactoring Backend is difficult --> need experienced developer
>  there and a specification
>      * Focusing in on the pain point first will give improvement performance 
I'd say a specification for *both* backend and frontend are def required. Specifically
when you're not skilled enough.
That would also help new contributor to jump in. 

>  
>  * Moving off the message driven backend would be good in order to make
>  the application more generic and not only Fedora centric.
>      * Potential to get more contributors from other communities.
>  
>  * Virtual Hackfest - September 13-15th
>      * Idea to sit down and look at the technical pain point. Slow
>  Requests, Exception in the consumer etc
>      * Write these down and come up with a plan to improve these. 
Why wait for Sept, 13?
I'd say let's start right now to dig this up.

To summarize:
In short term scenario, fix AMAP the slowness, while in the long term we start by
redesigning the API based on current call-flow w/ in mind an architecture that lets us
add/alter anything based on the redesign of the Front specification.

Cheers,
-x

[Clement Verna]

On Wed, 21 Aug 2019 at 20:21, Xavier トーマス LAMIEN <laxathom(a)fedoraproject.org&gt;
wrote:

>  > Hi all,
>
>  Sorry guy, I couldn't make it in time for the meeting.
>  My feedback below.
>
>  <wip>
>  >
>  > ## Notes
>  >
>  > * Frontend will be rewritten
>  > * Backend ? is this worth the rewrite ?
>  >     * Pyramid as a framework does not have a strong community and
>  adoption
>  >     * Django is the industry standard for Python based web application
>
>  I did audit the code of both API and the monolithic pyramid app after
>  flock and came to the same conclusion (besides some design choices which
>  don't make sense).
>  However, I would say not to rewrite the backend but to merge some of the
>  backend's badges management/call-flow w/in the API and provide only one
>  service.
>  The frontend will then relies on this API.
>  This will also provide to admins a better interface to manage badges
>  (awards, update, adding, promote, etc).
>  
Sounds good to me, I think that's a good trade-off

>  > * Current application is very slow
>  >     * Badges image are big
>
>  What I noticed is not all images are the same regarding sizing, ratio,
>  format (looks like more w/ the oldest).
>  Don't get me wrong, I'm talking about the image object, not the badge
>  itself
>
>  >     * Needs to download a lot of data to load a page
>  >     * If rewrite this need to be kept in mind
>
>  This is a design concerns I'd say. Also, need to take into account the
>  servers (including their location) which serve those files.
>  We will have to make some architecture choices on how/what to display
>  based on the page the end-user is on and the load the page requires.
>
>  >     * Python2 dependencies not maintained
>  > * Rewrite could also be porting current app to Django and slim down
>  > the number of pages/feature
>  >     * maybe 1 user page and 1 admin page.
>
>  +1 w/ Miro's comments
>  I'd be in favor of using AIOHTTP which provides pretty good efficiency
>  regarding server load/speed, good feature out-of-the-box and so much more.
>  I'd be glad to tell more about it if you're interested in for those who
>  don't know.
>  
That's also fine with me, we mentioned Django mostly because we wanted to
have a project with some traction. AIOHTTP is the main choice for async web
server in Python.

>  Also, we need to think about how an IT infrastructure will deploy
> such
>  services based on their architecture. How many instances, do they need a
>  reverse-proxy in front of them, the daily requests load, etc?
>  
 So the plan for badges is to be hosted on the community Openshift, there
is a current limit of number of pods (5) you can run there but I am pretty
sure this can be flexible for an app like badges.

>  >
>  > * Start with the frontend --using a JS framework and let the backend
>  > with pyramid but just serve JSON instead of templates.
>
>  As I said above, I'd be in favor of removing pyramid (even though I
>  support pyramid and was one of those who voted to have it as a selected
>  framework for infra dev)
>  But we need at some point to get thing right w/ the right tools when
>  needed and for what I say, I don't think Pyramid, Django would be a good
>  fit.
>
>  >
>  > * Refactoring Backend is difficult --> need experienced developer
>  > there and a specification
>  >     * Focusing in on the pain point first will give improvement
>  performance
>
>  I'd say a specification for *both* backend and frontend are def required.
>  Specifically when you're not skilled enough.
>  That would also help new contributor to jump in.
>  
+1
Some early design/spec has been started (see link below), I think it is
important to keep in sync between frontend and backend there.
https://docs.google.com/document/d/1r1wMi6IZretWuYg9PllHSyZXNZ9LpQIiiq5Kn...

>  >
>  > * Moving off the message driven backend would be good in order to make
>  > the application more generic and not only Fedora centric.
>  >     * Potential to get more contributors from other communities.
>  >
>  > * Virtual Hackfest - September 13-15th
>  >     * Idea to sit down and look at the technical pain point. Slow
>  > Requests, Exception in the consumer etc
>  >     * Write these down and come up with a plan to improve these.
>
>  Why wait for Sept, 13?
>  I'd say let's start right now to dig this up.
>  
Yeah \o/, I think the best is to try to coordinate effort on this list and
on #fedora-badges. That's where most the folks interested in helping are.


>  To summarize:
>  In short term scenario, fix AMAP the slowness, while in the long term we
>  start by redesigning the API based on current call-flow w/ in mind an
>  architecture that lets us add/alter anything based on the redesign of the
>  Front specification.
>  
Thanks for helping with this :-)

>  Cheers,
>  -x
>  _______________________________________________
>  Fedora Badges mailing list -- badges(a)lists.fedoraproject.org
>  To unsubscribe send an email to badges-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>