feedback requested for the data-export branch

[James Bowes]

Hi all:

Please take some time to read over the code in the data-export branch.
Relevant code lives in the org.fedoraproject.candlepin.sync package.

The exporter entry point is on ConsumerResource. The importer entry
point is on OwnerResource.

To create and consume your own export, try the following:

* Make your 'upstream' candlepin
  * run the deploy script with import dir and gendb set.
  * register a consumer, note the consumer's uuid
  * subscribe the consumer to some pools
  * download your export with:
    curl -k -u admin:admin https://localhost:8443/candlepin/consumers/$UUID/export > export.zip
* Turn your 'upstream' candlepin into your new 'downstream' candlepin
  * run the deploy script without import dir, but with gendb (to empty
    out the db)
  * import your export with:
    curl -u admin:admin -k -F export=@export.zip https://localhost:8443/candlepin/owners/1/import

Besides the new package, we also added an upstreamUuid field to Owner to
track consumer to owner mapping, and an upstreamPoolId to Subscription
to track the flow of upstream pool -> entitlement -> downstream
subscription. Why is Subscription not using upstreamEntitlementId? this
way, if you unentitle then reentitle your candlepin, the entitlement
object will have changed, but syncing will ignore it. Devan may wish to
point you at some of the work he did for products, specifically on the
ProductCurator.

Related doc - https://fedorahosted.org/candlepin/wiki/DataTransferFormat

Thanks!

-James

[Bryan Kearney]

On 07/09/2010 04:12 PM, James Bowes wrote:
> Hi all:
>
> Please take some time to read over the code in the data-export branch.
> Relevant code lives in the org.fedoraproject.candlepin.sync package.
>
> The exporter entry point is on ConsumerResource. The importer entry
> point is on OwnerResource.
>
> To create and consume your own export, try the following:
> Make your 'upstream' candlepin
> run the deploy script with import dir and gendb set.
> register a consumer, note the consumer's uuid
> subscribe the consumer to some pools
> download your export with:
> curl -k -u admin:admin https://localhost:8443/candlepin/consumers/$UUID/export%3E  export.zip
> Turn your 'upstream' candlepin into your new 'downstream' candlepin
> run the deploy script without import dir, but with gendb (to empty
> out the db)
> import your export with:
> curl -u admin:admin -k -F export=@export.zip https://localhost:8443/candlepin/owners/1/import
>
> Besides the new package, we also added an upstreamUuid field to Owner to
> track consumer to owner mapping, and an upstreamPoolId to Subscription
> to track the flow of upstream pool ->  entitlement ->  downstream
> subscription. Why is Subscription not using upstreamEntitlementId? this
> way, if you unentitle then reentitle your candlepin, the entitlement
> object will have changed, but syncing will ignore it. Devan may wish to
> point you at some of the work he did for products, specifically on the
> ProductCurator.

Why use a new field as opposed to the exsting external ID field? Dont 
know if I object, just curious.

-- bk

[James Bowes]

On Mon, Jul 12, 2010 at 01:53:57PM -0400, Bryan Kearney wrote:
> On 07/09/2010 04:12 PM, James Bowes wrote:
> > Hi all:
> >
> > Please take some time to read over the code in the data-export branch.
> > Relevant code lives in the org.fedoraproject.candlepin.sync package.
> >
> > The exporter entry point is on ConsumerResource. The importer entry
> > point is on OwnerResource.
> >
> > To create and consume your own export, try the following:
> > Make your 'upstream' candlepin
> > run the deploy script with import dir and gendb set.
> > register a consumer, note the consumer's uuid
> > subscribe the consumer to some pools
> > download your export with:
> > curl -k -u admin:admin https://localhost:8443/candlepin/consumers/$UUID/export%3E  export.zip
> > Turn your 'upstream' candlepin into your new 'downstream' candlepin
> > run the deploy script without import dir, but with gendb (to empty
> > out the db)
> > import your export with:
> > curl -u admin:admin -k -F export=@export.zip https://localhost:8443/candlepin/owners/1/import
> >
> > Besides the new package, we also added an upstreamUuid field to Owner to
> > track consumer to owner mapping, and an upstreamPoolId to Subscription
> > to track the flow of upstream pool ->  entitlement ->  downstream
> > subscription. Why is Subscription not using upstreamEntitlementId? this
> > way, if you unentitle then reentitle your candlepin, the entitlement
> > object will have changed, but syncing will ignore it. Devan may wish to
> > point you at some of the work he did for products, specifically on the
> > ProductCurator.
>
> Why use a new field as opposed to the exsting external ID field?
> Dont know if I object, just curious.

since no copy of the object exists locally, we could end up clashing
with ones that actually do exist locally. Also in both cases i think the
mapping that we added goes in the opposite direction from that is
already there.
> -- bk

-James

[Bryan Kearney]

On 07/09/2010 04:12 PM, James Bowes wrote:
> Hi all:
> Related doc - https://fedorahosted.org/candlepin/wiki/DataTransferFormat

Code Comments:

Exporter
    - Is there a race condition betweeb exporting the entitlements and 
the certificates and then products?? I assume given standard isolation 
there would be
         - Can we not hardcode the type MKT? Can we check to see if 
there is aproduct cert or some other means?
         - Export event?

metaexporter
    - Should the version be the CP version?
         - Do you think the exports should have a serial number?

ConsumerImporter
    - Do we want a way to force the UUID rewrite, or make them use the 
update function?

There is no entitlement Certificate Importer. I think we will need this 
for integration with pulp.

-- bk