Help reqest: dependencies for packaging writefreely

[Edd Salkield]

Hi all,

I'm currently working on packaging github.com/writeas/writefreely, a platform for building writing spaces.  Some of its dependencies are currently not packaged for Fedora, so I'm seeking some advice on the granularity to which I should be packaging each dependency.  I initially wrote this email to [e-mail], and I got directed here, so apologies if you're in both lists.

github.com/writeas/writefreely depends on github.com/writeas/web-core.  This is a library that is used by the project developers (write.as) for all their shared code.  It is likely that this will be used as a dependency for future packages (for example, when their photo sharing service, snap.as, gets released), but unlikely that it will ever be used by anyone other than these developers.

Therefore, should github.com/writeas/web-core be its own package, should I bundle it in with writefreely, should it be a subpackage of writefreely, or indeed something else?

Additionally, web-core has its own dependencies of the same kind.  Where possible, I have submitted PRs[1,2,3] to their repo to reduce unnecessary dependencies, but web-core still depends upon github.com/writeas/impart and github.com/writeas/openssl-go.  These libraries are both unlikely to be used anywhere else; should these be bundled too?

If the correct course of action is to bundle the dependencies (in a nested way), I would really appreciate somebody experienced at packaging Go packages to give some guidance on how to do this sensibly with the Go SPEC file macro system.  This is especially since the current documentation[4] seems pretty light on the topic of dealing with SPEC files with multiple sources.

Also, this is my first time joining a mailing list to actually get involved with the Fedora project, so do let me know if this sort of question actually belongs elsewhere.

Kind regards,
Edd Salkield

[1] https://github.com/writeas/web-core/pull/6
[2] https://github.com/writeas/web-core/pull/8
[3] https://github.com/writeas/web-core/pull/10
[4] https://docs.fedoraproject.org/en-US/packaging-guidelines/Golang/

[Robert-André Mauchin]

On Thursday, 24 September 2020 02:26:11 CEST Edd Salkield wrote:
> Hi all,
>
> I'm currently working on packaging github.com/writeas/writefreely, a
> platform for building writing spaces.  Some of its dependencies are
> currently not packaged for Fedora, so I'm seeking some advice on the
> granularity to which I should be packaging each dependency.  I initially
> wrote this email to [e-mail], and I got directed
> here, so apologies if you're in both lists.
> github.com/writeas/writefreely depends on github.com/writeas/web-core.  This
> is a library that is used by the project developers (write.as) for all
> their shared code.  It is likely that this will be used as a dependency for
> future packages (for example, when their photo sharing service, snap.as,
> gets released), but unlikely that it will ever be used by anyone other than
> these developers.
> Therefore, should github.com/writeas/web-core be its own package, should I
> bundle it in with writefreely, should it be a subpackage of writefreely, or
> indeed something else?
> Additionally, web-core has its own dependencies of the same kind.  Where
> possible, I have submitted PRs[1,2,3] to their repo to reduce unnecessary
> dependencies, but web-core still depends upon github.com/writeas/impart and
> github.com/writeas/openssl-go.  These libraries are both unlikely to be
> used anywhere else; should these be bundled too?
> If the correct course of action is to bundle the dependencies (in a nested
> way), I would really appreciate somebody experienced at packaging Go
> packages to give some guidance on how to do this sensibly with the Go SPEC
> file macro system.  This is especially since the current documentation[4]
> seems pretty light on the topic of dealing with SPEC files with multiple
> sources.
> Also, this is my first time joining a mailing list to actually get involved
> with the Fedora project, so do let me know if this sort of question
> actually belongs elsewhere.
> Kind regards,
> Edd Salkield
>
> [1] https://github.com/writeas/web-core/pull/6
> [2] https://github.com/writeas/web-core/pull/8
> [3] https://github.com/writeas/web-core/pull/10
> [4] https://docs.fedoraproject.org/en-US/packaging-guidelines/Golang/
> _______________________________________________
> golang mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List
> Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List
> Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
> g

So the policy is currently to unbundle everything.

This is the dependency path you are facing:

BuildRequires:  golang(github.com/gorilla/feeds)
BuildRequires:  golang(github.com/guregu/null)
BuildRequires:  golang(github.com/ikeikeikeike/go-sitemap-generator/v2/stm)
    BuildRequires:  golang(github.com/clbanning/mxj)
BuildRequires:  golang(github.com/manifoldco/promptui)
BuildRequires:  golang(github.com/writeas/activity/streams)
    BuildRequires:  golang(github.com/go-fed/httpsig)
BuildRequires:  golang(github.com/writeas/activityserve)
    BuildRequires:  golang(github.com/go-fed/httpsig)
    BuildRequires:  golang(github.com/writefreely/go-nodeinfo)
        BuildRequires:  golang(github.com/writeas/go-webfinger)
            BuildRequires:  golang(github.com/captncraig/cors)
    BuildRequires:  golang(github.com/gologme/log)
BuildRequires:  golang(github.com/writeas/go-strip-markdown)
BuildRequires:  golang(github.com/writeas/httpsig)
BuildRequires:  golang(github.com/writeas/import)
    BuildRequires:  golang(github.com/writeas/go-writeas/v2)
        BuildRequires:  golang(github.com/writeas/impart)
        BuildRequires:  golang(code.as/core/socks)
BuildRequires:  golang(github.com/writeas/monday)
    Need to add alternative import path to our golang-github-fatih-set
BuildRequires:  golang(github.com/writeas/slug)
BuildRequires:  golang(github.com/writeas/web-core)
    BuildRequires:  golang(github.com/kylemcc/twitter-text-go/extract)
    BuildRequires:  golang(github.com/nu7hatch/gouuid)
    BuildRequires:  golang(github.com/writeas/impart)
    BuildRequires:  golang(github.com/writeas/nerds)
    BuildRequires:  golang(github.com/writeas/saturday)

Around 25 new packages.
> Therefore, should github.com/writeas/web-core be its own package

Yes
> should I bundle it in with writefree

Maybe but it is strongly not recommended.
> should it be a subpackage of writefreely

No.
> but web-core still depends upon github.com/writeas/impart and
> github.com/writeas/openssl-go

github.com/writeas/openssl-go seems to have gone away in the latest version.
github.com/writeas/impart is also a direct dep of github.com/writeas/writefreely 
so there is no way around.
> If the correct course of action is to bundle the dependencies (in a nested
> way), I would really appreciate somebody experienced at packaging Go
> packages to give some guidance on how to do this sensibly with the Go SPEC
> file macro system.  This is especially since the current documentation[4]
> seems pretty light on the topic of dealing with SPEC files with multiple
> sources.

You git clone the repo, checkout the last version, run go mod vendor
tar.gz the vendor/ subdirectory
add this archive to you spec:

==========================================================================
# Generated by go2rpm 2
%bcond_without check

# https://github.com/writeas/writefreely
%global goipath         github.com/writeas/writefreely
Version:                0.12.0

%gometa

%global common_description %{expand:
Build a digital writing community.}

%global golicenses      LICENSE
%global godocs          AUTHORS.md CONTRIBUTING.md README.md

Name:           writefreely
Release:        1%{?dist}
Summary:        Build a digital writing community

# Here you must add all the bundled packages licenses
License:        AGPLv3
URL:            %{gourl}
Source0:        %{gosource}
# From go mod vendor
Source1:        vendor-%{version}.tar.gz

Provides:  bundled(golang(github.com/dustin/go-humanize))
Provides:  bundled(golang(github.com/fatih/color))
Provides:  bundled(golang(github.com/go-sql-driver/mysql))
Provides:  bundled(golang(github.com/gorilla/feeds))
Provides:  bundled(golang(github.com/gorilla/mux))
Provides:  bundled(golang(github.com/gorilla/schema))
Provides:  bundled(golang(github.com/gorilla/sessions))
Provides:  bundled(golang(github.com/guregu/null))
Provides:  bundled(golang(github.com/guregu/null/zero))
Provides:  bundled(golang(github.com/hashicorp/go-multierror))
Provides:  bundled(golang(github.com/ikeikeikeike/go-sitemap-generator/v2/stm))
Provides:  bundled(golang(github.com/kylemcc/twitter-text-go/extract))
Provides:  bundled(golang(github.com/manifoldco/promptui))
Provides:  bundled(golang(github.com/microcosm-cc/bluemonday))
Provides:  bundled(golang(github.com/mitchellh/go-wordwrap))
Provides:  bundled(golang(github.com/nu7hatch/gouuid))
Provides:  bundled(golang(github.com/urfave/cli/v2))
Provides:  bundled(golang(github.com/writeas/activity/streams))
Provides:  bundled(golang(github.com/writeas/activityserve))
Provides:  bundled(golang(github.com/writeas/go-strip-markdown))
Provides:  bundled(golang(github.com/writeas/go-webfinger))
Provides:  bundled(golang(github.com/writeas/httpsig))
Provides:  bundled(golang(github.com/writeas/impart))
Provides:  bundled(golang(github.com/writeas/import))
Provides:  bundled(golang(github.com/writeas/monday))
Provides:  bundled(golang(github.com/writeas/nerds/store))
Provides:  bundled(golang(github.com/writeas/saturday))
Provides:  bundled(golang(github.com/writeas/slug))
Provides:  bundled(golang(github.com/writeas/web-core/activitypub))
Provides:  bundled(golang(github.com/writeas/web-core/activitystreams))
Provides:  bundled(golang(github.com/writeas/web-core/auth))
Provides:  bundled(golang(github.com/writeas/web-core/bots))
Provides:  bundled(golang(github.com/writeas/web-core/converter))
Provides:  bundled(golang(github.com/writeas/web-core/data))
Provides:  bundled(golang(github.com/writeas/web-core/i18n))
Provides:  bundled(golang(github.com/writeas/web-core/id))
Provides:  bundled(golang(github.com/writeas/web-core/l10n))
Provides:  bundled(golang(github.com/writeas/web-core/log))
Provides:  bundled(golang(github.com/writeas/web-core/memo))
Provides:  bundled(golang(github.com/writeas/web-core/passgen))
Provides:  bundled(golang(github.com/writeas/web-core/posts))
Provides:  bundled(golang(github.com/writeas/web-core/query))
Provides:  bundled(golang(github.com/writeas/web-core/stringmanip))
Provides:  bundled(golang(github.com/writeas/web-core/tags))
Provides:  bundled(golang(github.com/writefreely/go-nodeinfo))
Provides:  bundled(golang(golang.org/x/crypto/acme/autocert))
Provides:  bundled(golang(gopkg.in/ini.v1))
Provides:  bundled(golang(github.com/stretchr/testify/assert))

BuildRequires:  go-bindata
BuildRequires:  sqlite-devel
BuildRequires:  nodejs-less

%description
%{common_description}

%prep
%goprep -k
%setup -q -T -D -a 1 -n %{extractdir}

%build
Add build steps here (bindata and lessc part)

go-bindata -pkg writefreely -ignore=\.gitignore -tags="!wflib" schema.sql sqlite.sql
pushd less/
%make_build all
popd

for cmd in cmd/* ; do
  %gobuild -tags='sqlite -o %{gobuilddir}/bin/$(basename $cmd) %{goipath}/$cmd
done

%install
install -m 0755 -vd                     %{buildroot}%{_bindir}
install -m 0755 -vp %{gobuilddir}/bin/* %{buildroot}%{_bindir}/

%if %{with check}
%check
%gocheck
%endif

%files
%license LICENSE
%doc AUTHORS.md CONTRIBUTING.md README.md
%{_bindir}/*

%changelog

==========================================================================

In any case, the building of the app is gonna be tough as it has a special 
build script that won't use Fedora flags. There some documentation here: 
https://writefreely.org/docs/latest/developer/setup about how to reproduce 
the build steps.

The nodejs part won't work out of the box as it requires d/l less and Koji/Mock 
don't have Internet support while building.

Our less package https://src.fedoraproject.org/rpms/nodejs-less has been orphaned
so you might need to revive it. It failed the last mass rebuild (FTBFS).

Another issue is that the binary expect these dir to be in the same directory as the binary

cp -r templates $(BUILDPATH)
    cp -r pages $(BUILDPATH)
    cp -r static $(BUILDPATH)
    mkdir $(BUILDPATH)/keys

whereas they should be in datadir/ for a distro. Arch provides a config.ini for this, take a look at how they do things:
https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=writefreely
https://aur.archlinux.org/packages/writefreely/

This is not a simple package, I strongly suggest working in a COPR to test you build scripts.

[Edd Salkield]

Thanks for taking the time to give me a comprehensive response.  I was rather hoping this would be a one-evening project, which now seems rather naive indeed.  I'll work through the core dependencies first and see how I get on.