Lazy Cert Regeneration Design

[Devan Goodwin]

There are a few ways certs get regenerated:

1. During refresh pools if
   (a) the product name/attributes change
   (b) provided products are added/removed
   (c) subscription dates changed
   (d) subscription was revoked
   - quantity changes could lead to certs getting revoked if we're over 
the limit, but does not regenerate anything.
   - we do not regenerate on content set changes (we do for provided 
products, but not just content sets within one of those provided 
products). we do not store content data locally in some deployments so 
there's nothing to compare against to see if anything changed.
   - I am assuming the business rules would require us to immediately 
revoke for (d), but to use lazy revocation for all of the others.

2. An explicit call to regenerate everything for some product, I do not 
know where/if IT uses this: PUT /entitlements/product/{productId} Lazy 
regeneration should probably be used here. (doesn't matter what changed, 
we don't really know, but we can rest assured it wasn't anyone losing 
their subscription access)

3. An explicit call to regenerate everything for a given consumer: PUT 
/consumers/{uuid}/certificates. IMO we should not do any lazy 
regeneration here, I think the use case for this call was for security 
reasons and the like.

4. If configured for environments, whenever content is promoted/demoted 
to an environment, we mass regenerate for those consumers. Lazy regen 
seems good here.



This seems fairly straightforward so far. The request was for 
distributor consumer types only.

- Add a "dirty" column to cp_entitlement, flip it to true as desired for 
entitlements when they need regeneration during (1) or (2) above.

- In GET /consumers/{uuid}/export check all entitlements for the dirty 
flag, if any are found regenerate them, then proceed normally.

jbowes has proposed possibly leveraging lazy regen for all consumers of 
all types for some cert v2 purposes. (might need more info from you on 
this jbowes) I think this just implies expanding the hooks for 
triggering regeneration to:

GET /consumers/{uuid}
GET /consumers/{uuid}/entitlements
GET /consumers/{uuid}/certificates (both JSON and ZIP calls)
GET /consumers/{uuid}/certificates/serials

I don't think this is much more work, the method to scan for dirty 
entitlements and regenerate would be shared across all these API calls. 
The implementation is cleaner particularly in the locations where we 
decide to set dirty flag or not, if this is not specific to just one 
consumer type and path through the code.

Thoughts?

Thanks,

Devan

[Bryan Kearney]

On 05/24/2012 03:30 PM, Devan Goodwin wrote:
>  There are a few ways certs get regenerated:
>
>  1. During refresh pools if
>  (a) the product name/attributes change
>  (b) provided products are added/removed
>  (c) subscription dates changed
>  (d) subscription was revoked
>  - quantity changes could lead to certs getting revoked if we're over the
>  limit, but does not regenerate anything.
>  - we do not regenerate on content set changes (we do for provided
>  products, but not just content sets within one of those provided
>  products). we do not store content data locally in some deployments so
>  there's nothing to compare against to see if anything changed. 
Isnt this why we had a refresh based on product? If we dont have this, I 
have a new backlog item :)

>  - I am assuming the business rules would require us to immediately
>  revoke for (d), but to use lazy revocation for all of the others. 
Agreed.

>  2. An explicit call to regenerate everything for some product, I do not
>  know where/if IT uses this: PUT /entitlements/product/{productId} Lazy
>  regeneration should probably be used here. (doesn't matter what changed,
>  we don't really know, but we can rest assured it wasn't anyone losing
>  their subscription access) 

agreed... answers my question from above. I thought we had this already.

>  3. An explicit call to regenerate everything for a given consumer: PUT
>  /consumers/{uuid}/certificates. IMO we should not do any lazy
>  regeneration here, I think the use case for this call was for security
>  reasons and the like. 
No, I would here as well. Or, give an option:

PUT  /consumers/{uuid}/certificates?lazy=HellNo

>  4. If configured for environments, whenever content is promoted/demoted
>  to an environment, we mass regenerate for those consumers. Lazy regen
>  seems good here. 
Need to define lazy for me here. I would assume we would not have

distributors and environments.

>  This seems fairly straightforward so far. The request was for
>  distributor consumer types only.
>
>  - Add a "dirty" column to cp_entitlement, flip it to true as desired for
>  entitlements when they need regeneration during (1) or (2) above.
>
>  - In GET /consumers/{uuid}/export check all entitlements for the dirty
>  flag, if any are found regenerate them, then proceed normally.
>
>  jbowes has proposed possibly leveraging lazy regen for all consumers of
>  all types for some cert v2 purposes. (might need more info from you on
>  this jbowes) I think this just implies expanding the hooks for
>  triggering regeneration to: 
I agree. It may be nice to have the pain of creating the new certs at 
next machine checkin. This would speed up the repool by spreading out 
the pain.

>  GET /consumers/{uuid}
>  GET /consumers/{uuid}/entitlements
>  GET /consumers/{uuid}/certificates (both JSON and ZIP calls)
>  GET /consumers/{uuid}/certificates/serials
>
>  I don't think this is much more work, the method to scan for dirty
>  entitlements and regenerate would be shared across all these API calls.
>  The implementation is cleaner particularly in the locations where we
>  decide to set dirty flag or not, if this is not specific to just one
>  consumer type and path through the code.
>
>  Thoughts?
>
>  Thanks, 
-- bk

[Devan Goodwin]

On Fri 25 May 2012 09:54:20 AM ADT, Bryan Kearney wrote:
>  On 05/24/2012 03:30 PM, Devan Goodwin wrote:
> > There are a few ways certs get regenerated:
> >
> > 1. During refresh pools if
> > (a) the product name/attributes change
> > (b) provided products are added/removed
> > (c) subscription dates changed
> > (d) subscription was revoked
> > - quantity changes could lead to certs getting revoked if we're over the
> > limit, but does not regenerate anything.
> > - we do not regenerate on content set changes (we do for provided
> > products, but not just content sets within one of those provided
> > products). we do not store content data locally in some deployments so
> > there's nothing to compare against to see if anything changed.
>
>  Isnt this why we had a refresh based on product? If we dont have this,
>  I have a new backlog item :)
>
> > - I am assuming the business rules would require us to immediately
> > revoke for (d), but to use lazy revocation for all of the others.
>
>  Agreed.
>
> >
> > 2. An explicit call to regenerate everything for some product, I do not
> > know where/if IT uses this: PUT /entitlements/product/{productId} Lazy
> > regeneration should probably be used here. (doesn't matter what changed,
> > we don't really know, but we can rest assured it wasn't anyone losing
> > their subscription access)
>
>
>  agreed... answers my question from above. I thought we had this already.
>
> >
> > 3. An explicit call to regenerate everything for a given consumer: PUT
> > /consumers/{uuid}/certificates. IMO we should not do any lazy
> > regeneration here, I think the use case for this call was for security
> > reasons and the like.
>
>  No, I would here as well. Or, give an option:
>
>  PUT /consumers/{uuid}/certificates?lazy=HellNo 
Ok if there's no known use case for demanding immediate regen for a 
consumer's certs, then may as well make it default to lazy and stay 
consistent with the rest of the API.

> >
> > 4. If configured for environments, whenever content is promoted/demoted
> > to an environment, we mass regenerate for those consumers. Lazy regen
> > seems good here.
>  Need to define lazy for me here. I would assume we would not have
>  distributors and environments.
>  
Only in like a nested Katello scenario, but my hope is that we can 
proceed with this for all consumers, not just distributors. So when 
content is promoted and demoted, all affected consumers would get their 
entitlements dirty flag set and will regen when they next check in. This 
actually alleviates one of the big concerns I had after the environments 
work where promotion/demotion could be really slow. Now, you can 
promote/demote multiple times without triggering any mass regen, should 
work a lot better actually.

> >
> >
> >
> > This seems fairly straightforward so far. The request was for
> > distributor consumer types only.
> >
> > - Add a "dirty" column to cp_entitlement, flip it to true as desired for
> > entitlements when they need regeneration during (1) or (2) above.
> >
> > - In GET /consumers/{uuid}/export check all entitlements for the dirty
> > flag, if any are found regenerate them, then proceed normally.
> >
> > jbowes has proposed possibly leveraging lazy regen for all consumers of
> > all types for some cert v2 purposes. (might need more info from you on
> > this jbowes) I think this just implies expanding the hooks for
> > triggering regeneration to:
>
>  I agree. It may be nice to have the pain of creating the new certs at
>  next machine checkin. This would speed up the repool by spreading out
>  the pain. 
Cool, I am liking the sound of it as well.

Thanks,

Devan

> >
> > GET /consumers/{uuid}
> > GET /consumers/{uuid}/entitlements
> > GET /consumers/{uuid}/certificates (both JSON and ZIP calls)
> > GET /consumers/{uuid}/certificates/serials
> >
> > I don't think this is much more work, the method to scan for dirty
> > entitlements and regenerate would be shared across all these API calls.
> > The implementation is cleaner particularly in the locations where we
> > decide to set dirty flag or not, if this is not specific to just one
> > consumer type and path through the code.
> >
> > Thoughts?
> >
> > Thanks,
>
>  -- bk
>
>
>  _______________________________________________
>  candlepin mailing list
>  candlepin(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/candlepin

[Bryan Kearney]

On 05/25/2012 10:37 AM, Devan Goodwin wrote:
>  On Fri 25 May 2012 09:54:20 AM ADT, Bryan Kearney wrote:
> > On 05/24/2012 03:30 PM, Devan Goodwin wrote:
> >> There are a few ways certs get regenerated:
> >>
> >> 1. During refresh pools if
> >> (a) the product name/attributes change
> >> (b) provided products are added/removed
> >> (c) subscription dates changed
> >> (d) subscription was revoked
> >> - quantity changes could lead to certs getting revoked if we're over the
> >> limit, but does not regenerate anything.
> >> - we do not regenerate on content set changes (we do for provided
> >> products, but not just content sets within one of those provided
> >> products). we do not store content data locally in some deployments so
> >> there's nothing to compare against to see if anything changed.
> >
> > Isnt this why we had a refresh based on product? If we dont have this,
> > I have a new backlog item :)
> >
> >> - I am assuming the business rules would require us to immediately
> >> revoke for (d), but to use lazy revocation for all of the others.
> >
> > Agreed.
> >
> >>
> >> 2. An explicit call to regenerate everything for some product, I do not
> >> know where/if IT uses this: PUT /entitlements/product/{productId} Lazy
> >> regeneration should probably be used here. (doesn't matter what changed,
> >> we don't really know, but we can rest assured it wasn't anyone losing
> >> their subscription access)
> >
> >
> > agreed... answers my question from above. I thought we had this already. 
Do we have this already? If not, I would want to add it to the backlog.
> >
> >>
> >> 3. An explicit call to regenerate everything for a given consumer: PUT
> >> /consumers/{uuid}/certificates. IMO we should not do any lazy
> >> regeneration here, I think the use case for this call was for security
> >> reasons and the like.
> >
> > No, I would here as well. Or, give an option:
> >
> > PUT /consumers/{uuid}/certificates?lazy=HellNo
>
>  Ok if there's no known use case for demanding immediate regen for a
>  consumer's certs, then may as well make it default to lazy and stay
>  consistent with the rest of the API.
>
> >
> >>
> >> 4. If configured for environments, whenever content is promoted/demoted
> >> to an environment, we mass regenerate for those consumers. Lazy regen
> >> seems good here.
> > Need to define lazy for me here. I would assume we would not have
> > distributors and environments.
> >
>
>  Only in like a nested Katello scenario, but my hope is that we can
>  proceed with this for all consumers, not just distributors. So when
>  content is promoted and demoted, all affected consumers would get their
>  entitlements dirty flag set and will regen when they next check in. This
>  actually alleviates one of the big concerns I had after the environments
>  work where promotion/demotion could be really slow. Now, you can
>  promote/demote multiple times without triggering any mass regen, should
>  work a lot better actually.
>
> >>
> >>
> >>
> >> This seems fairly straightforward so far. The request was for
> >> distributor consumer types only.
> >>
> >> - Add a "dirty" column to cp_entitlement, flip it to true as desired
> for
> >> entitlements when they need regeneration during (1) or (2) above.
> >>
> >> - In GET /consumers/{uuid}/export check all entitlements for the dirty
> >> flag, if any are found regenerate them, then proceed normally.
> >>
> >> jbowes has proposed possibly leveraging lazy regen for all consumers of
> >> all types for some cert v2 purposes. (might need more info from you on
> >> this jbowes) I think this just implies expanding the hooks for
> >> triggering regeneration to:
> >
> > I agree. It may be nice to have the pain of creating the new certs at
> > next machine checkin. This would speed up the repool by spreading out
> > the pain.
>
>  Cool, I am liking the sound of it as well.
>
>  Thanks,
>
>  Devan
>
> >
> >>
> >> GET /consumers/{uuid}
> >> GET /consumers/{uuid}/entitlements
> >> GET /consumers/{uuid}/certificates (both JSON and ZIP calls)
> >> GET /consumers/{uuid}/certificates/serials
> >>
> >> I don't think this is much more work, the method to scan for dirty
> >> entitlements and regenerate would be shared across all these API calls.
> >> The implementation is cleaner particularly in the locations where we
> >> decide to set dirty flag or not, if this is not specific to just one
> >> consumer type and path through the code.
> >>
> >> Thoughts?
> >>
> >> Thanks,
> >
> > -- bk
> >
> >
> > _______________________________________________
> > candlepin mailing list
> > candlepin(a)lists.fedorahosted.org
> > https://fedorahosted.org/mailman/listinfo/candlepin
>

[Devan Goodwin]

On Fri, May 25, 2012 at 11:38 AM, Bryan Kearney <bkearney(a)redhat.com&gt; wrote:

> >>>
> >>> 2. An explicit call to regenerate everything for some product, I do not
> >>> know where/if IT uses this: PUT /entitlements/product/{productId} Lazy
> >>> regeneration should probably be used here. (doesn't matter what changed,
> >>> we don't really know, but we can rest assured it wasn't anyone
> losing
> >>> their subscription access)
> >>
> >>
> >>
> >> agreed... answers my question from above. I thought we had this already.
>
>
>  Do we have this already? If not, I would want to add it to the backlog. 
Yup we have it already, all of these are existing paths where certs
can get regenerated.

[Bryan Kearney]

On 05/25/2012 10:37 AM, Devan Goodwin wrote:
>  On Fri 25 May 2012 09:54:20 AM ADT, Bryan Kearney wrote:
> > On 05/24/2012 03:30 PM, Devan Goodwin wrote:
> >> There are a few ways certs get regenerated:
> >>
> >> 1. During refresh pools if
> >> (a) the product name/attributes change
> >> (b) provided products are added/removed
> >> (c) subscription dates changed
> >> (d) subscription was revoked
> >> - quantity changes could lead to certs getting revoked if we're over the
> >> limit, but does not regenerate anything.
> >> - we do not regenerate on content set changes (we do for provided
> >> products, but not just content sets within one of those provided
> >> products). we do not store content data locally in some deployments so
> >> there's nothing to compare against to see if anything changed.
> >
> > Isnt this why we had a refresh based on product? If we dont have this,
> > I have a new backlog item :)
> >
> >> - I am assuming the business rules would require us to immediately
> >> revoke for (d), but to use lazy revocation for all of the others.
> >
> > Agreed.
> >
> >>
> >> 2. An explicit call to regenerate everything for some product, I do not
> >> know where/if IT uses this: PUT /entitlements/product/{productId} Lazy
> >> regeneration should probably be used here. (doesn't matter what changed,
> >> we don't really know, but we can rest assured it wasn't anyone losing
> >> their subscription access)
> >
> >
> > agreed... answers my question from above. I thought we had this already.
> >
> >>
> >> 3. An explicit call to regenerate everything for a given consumer: PUT
> >> /consumers/{uuid}/certificates. IMO we should not do any lazy
> >> regeneration here, I think the use case for this call was for security
> >> reasons and the like.
> >
> > No, I would here as well. Or, give an option:
> >
> > PUT /consumers/{uuid}/certificates?lazy=HellNo
>
>  Ok if there's no known use case for demanding immediate regen for a
>  consumer's certs, then may as well make it default to lazy and stay
>  consistent with the rest of the API.
>
> >
> >>
> >> 4. If configured for environments, whenever content is promoted/demoted
> >> to an environment, we mass regenerate for those consumers. Lazy regen
> >> seems good here.
> > Need to define lazy for me here. I would assume we would not have
> > distributors and environments.
> >
>
>  Only in like a nested Katello scenario, but my hope is that we can
>  proceed with this for all consumers, not just distributors. So when
>  content is promoted and demoted, all affected consumers would get their
>  entitlements dirty flag set and will regen when they next check in. This
>  actually alleviates one of the big concerns I had after the environments
>  work where promotion/demotion could be really slow. Now, you can
>  promote/demote multiple times without triggering any mass regen, should
>  work a lot better actually. 
Agree... that makes sense.

-- bk

[James Bowes]

On Fri, May 25, 2012 at 08:54:20AM -0400, Bryan Kearney wrote:
>  On 05/24/2012 03:30 PM, Devan Goodwin wrote:
>  >There are a few ways certs get regenerated:
>  >
>  >1. During refresh pools if
>  >(a) the product name/attributes change
>  >(b) provided products are added/removed
>  >(c) subscription dates changed
>  >(d) subscription was revoked
>  >- quantity changes could lead to certs getting revoked if we're over the
>  >limit, but does not regenerate anything.
>  >- we do not regenerate on content set changes (we do for provided
>  >products, but not just content sets within one of those provided
>  >products). we do not store content data locally in some deployments so
>  >there's nothing to compare against to see if anything changed.
>  
>  Isnt this why we had a refresh based on product? If we dont have
>  this, I have a new backlog item :)
>  
>  >- I am assuming the business rules would require us to immediately
>  >revoke for (d), but to use lazy revocation for all of the others.
>  
>  Agreed.
>  
>  >
>  >2. An explicit call to regenerate everything for some product, I do not
>  >know where/if IT uses this: PUT /entitlements/product/{productId} Lazy
>  >regeneration should probably be used here. (doesn't matter what changed,
>  >we don't really know, but we can rest assured it wasn't anyone losing
>  >their subscription access)
>  
>  
>  agreed... answers my question from above. I thought we had this already.
>   
We have it, but it might not be used by anyone.

>  >
>  >3. An explicit call to regenerate everything for a given consumer: PUT
>  >/consumers/{uuid}/certificates. IMO we should not do any lazy
>  >regeneration here, I think the use case for this call was for security
>  >reasons and the like.
>  
>  No, I would here as well. Or, give an option:
>  
>  PUT  /consumers/{uuid}/certificates?lazy=HellNo
>  
>  >
>  >4. If configured for environments, whenever content is promoted/demoted
>  >to an environment, we mass regenerate for those consumers. Lazy regen
>  >seems good here.
>  Need to define lazy for me here. I would assume we would not have
>  distributors and environments.
>  
>  >
>  >
>  >
>  >This seems fairly straightforward so far. The request was for
>  >distributor consumer types only.
>  >
>  >- Add a "dirty" column to cp_entitlement, flip it to true as desired for
>  >entitlements when they need regeneration during (1) or (2) above.
>  >
>  >- In GET /consumers/{uuid}/export check all entitlements for the dirty
>  >flag, if any are found regenerate them, then proceed normally.
>  >
>  >jbowes has proposed possibly leveraging lazy regen for all consumers of
>  >all types for some cert v2 purposes. (might need more info from you on
>  >this jbowes) I think this just implies expanding the hooks for
>  >triggering regeneration to:
>  
>  I agree. It may be nice to have the pain of creating the new certs
>  at next machine checkin. This would speed up the repool by spreading
>  out the pain.
>  
>  >
>  >GET /consumers/{uuid}
>  >GET /consumers/{uuid}/entitlements
>  >GET /consumers/{uuid}/certificates (both JSON and ZIP calls)
>  >GET /consumers/{uuid}/certificates/serials
>  >
>  >I don't think this is much more work, the method to scan for dirty
>  >entitlements and regenerate would be shared across all these API calls.
>  >The implementation is cleaner particularly in the locations where we
>  >decide to set dirty flag or not, if this is not specific to just one
>  >consumer type and path through the code.
>  >
>  >Thoughts?
>  >
>  >Thanks,
>  
>  -- bk
>  
>  
>  _______________________________________________
>  candlepin mailing list
>  candlepin(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/candlepin 
-James

[James Bowes]

On Thu, May 24, 2012 at 04:30:32PM -0300, Devan Goodwin wrote:
<snip> 
>  jbowes has proposed possibly leveraging lazy regen for all consumers
>  of all types for some cert v2 purposes. (might need more info from
>  you on this jbowes) I think this just implies expanding the hooks
>  for triggering regeneration to:
>  
>  GET /consumers/{uuid}
>  GET /consumers/{uuid}/entitlements
>  GET /consumers/{uuid}/certificates (both JSON and ZIP calls)
>  GET /consumers/{uuid}/certificates/serials
>  
>  I don't think this is much more work, the method to scan for dirty
>  entitlements and regenerate would be shared across all these API
>  calls. The implementation is cleaner particularly in the locations
>  where we decide to set dirty flag or not, if this is not specific to
>  just one consumer type and path through the code.
>  
>  Thoughts?
>  
My thinking on making regens lazy for all consumer types:
 1. spread x509 related load out across time. systems check in every 4
    hours or whatever, not all at once.
 2. with multiple certificate formats, there's no need to know what
    version the consumer wants before they ask for it. we simply
    regenerate what they ask for (via api or looking at a header on the
    request or whatever).
 3. fewer special cases make for cleaner code
 4. if we want to be nice to distributors and let them use certs that
    aren't exactly current, shouldn't we extend the same courtesy to system
    consumers?

Doing a lazy regen implies that the real work of deciding if a
certificate is still good is when we put that old certificate on the CRL
list. We'd only add a certificate to the CRL immediately if it has
recieved a detectable net loss in access to content, like:
- quantity of entitlement has decreased (not sure if this is a real
  thing or not)
- content set is removed from a product (not detectable?)
- content set url is changed on a product (not detectable?)
- product is removed from a subscription

If we hit one of those cases on a refreshpools, we add the cert cerial
to the CRL, and mark it as dirty, letting the client trigger the regen
at some later point.

If we don't hit one of those cases, like if we've added a new product,
we simply mark the cert as dirty. That dirty cert is still valid until
the new  regen is done, but why not just keep it valid for as long as
the newly regenerated version? that is, can we reuse the cert serial on
the new cert, keeping the old version still valid, and reducing both the
clutter in our CRL and the rate at which we consume cert serials? I
think that would work.


>  Thanks,
>  
>  Devan
>  _______________________________________________
>  candlepin mailing list
>  candlepin(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/candlepin 
-James

[Devan Goodwin]

On 05/25/2012 02:50 PM, James Bowes wrote:
>  On Thu, May 24, 2012 at 04:30:32PM -0300, Devan Goodwin wrote:
>  <snip>
> > jbowes has proposed possibly leveraging lazy regen for all consumers
> > of all types for some cert v2 purposes. (might need more info from
> > you on this jbowes) I think this just implies expanding the hooks
> > for triggering regeneration to:
> >
> > GET /consumers/{uuid}
> > GET /consumers/{uuid}/entitlements
> > GET /consumers/{uuid}/certificates (both JSON and ZIP calls)
> > GET /consumers/{uuid}/certificates/serials
> >
> > I don't think this is much more work, the method to scan for dirty
> > entitlements and regenerate would be shared across all these API
> > calls. The implementation is cleaner particularly in the locations
> > where we decide to set dirty flag or not, if this is not specific to
> > just one consumer type and path through the code.
> >
> > Thoughts?
> >
>
>  My thinking on making regens lazy for all consumer types:
>    1. spread x509 related load out across time. systems check in every 4
>       hours or whatever, not all at once.
>    2. with multiple certificate formats, there's no need to know what
>       version the consumer wants before they ask for it. we simply
>       regenerate what they ask for (via api or looking at a header on the
>       request or whatever).
>    3. fewer special cases make for cleaner code
>    4. if we want to be nice to distributors and let them use certs that
>       aren't exactly current, shouldn't we extend the same courtesy to system
>       consumers?
>  
+1 to all.

>  Doing a lazy regen implies that the real work of deciding if a
>  certificate is still good is when we put that old certificate on the CRL
>  list. We'd only add a certificate to the CRL immediately if it has
>  recieved a detectable net loss in access to content, like:
>  - quantity of entitlement has decreased (not sure if this is a real
>     thing or not)
>  - content set is removed from a product (not detectable?)
>  - content set url is changed on a product (not detectable?)
>  - product is removed from a subscription
>
>  If we hit one of those cases on a refreshpools, we add the cert cerial
>  to the CRL, and mark it as dirty, letting the client trigger the regen
>  at some later point. 
Quantity decreasing should probably do as it does now and go ahead and 
revoke, this isn't really a regeneration but an outright revocation, and 
probably is not important for the use case we're trying to solve. I vote 
we let this immediately go to CRL and revoke the cert as we do today. 
(it's probably a pretty rare situation anyhow)

Content set changes are indeed not detectable, but possibly being 
triggered by IT to mass regenerate via that product specific API call. 
We will not know why we're being triggered though so as it is today, we 
cannot decide if we're going to CRL now or later when we regenerate. 
However as per next point I am hoping we can keep it simple and have 
only two situations.

For products being removed I will wager we still do not want to add to 
the CRL immediately, driver here was to not invalidate distributor 
certificates and my guess is we should not be doing that even if we're 
removing access to a particular repo from their subscription. Can we get 
clarification on this? (should we immediately invalidate certificates if 
a product is removed from a subscription)

Hopefully we can just stay with these two situations:

1. we're either adding to CRL + revoking (overconsumption, normal 
unbind, no work required because this is how it is today)

2. we're flagging entitlement dirty now, and adding to CRL + 
regenerating on next check-in or manifest generation

>  If we don't hit one of those cases, like if we've added a new product,
>  we simply mark the cert as dirty. That dirty cert is still valid until
>  the new  regen is done, but why not just keep it valid for as long as
>  the newly regenerated version? that is, can we reuse the cert serial on
>  the new cert, keeping the old version still valid, and reducing both the
>  clutter in our CRL and the rate at which we consume cert serials? I
>  think that would work. 
Having multiple certificates out in the wild with different contents but 
the same serial, all still valid, seems strange and like a probable 
source of future confusion, and perhaps an issue for our database. Each 
unique cert seems like it should probably have it's own serial to me.

If CRL clutter is a current known problem then it's worth considering, 
but otherwise I'd propose we leave that for another day and solve it if 
the issue arises.

Cheers,

Devan

[James Bowes]

On Fri, May 25, 2012 at 03:43:00PM -0300, Devan Goodwin wrote:
>  On 05/25/2012 02:50 PM, James Bowes wrote:
>  >On Thu, May 24, 2012 at 04:30:32PM -0300, Devan Goodwin wrote:
>  ><snip>
>  >>jbowes has proposed possibly leveraging lazy regen for all consumers
>  >>of all types for some cert v2 purposes. (might need more info from
>  >>you on this jbowes) I think this just implies expanding the hooks
>  >>for triggering regeneration to:
>  >>
>  >>GET /consumers/{uuid}
>  >>GET /consumers/{uuid}/entitlements
>  >>GET /consumers/{uuid}/certificates (both JSON and ZIP calls)
>  >>GET /consumers/{uuid}/certificates/serials
>  >>
>  >>I don't think this is much more work, the method to scan for dirty
>  >>entitlements and regenerate would be shared across all these API
>  >>calls. The implementation is cleaner particularly in the locations
>  >>where we decide to set dirty flag or not, if this is not specific to
>  >>just one consumer type and path through the code.
>  >>
>  >>Thoughts?
>  >>
>  >
>  >My thinking on making regens lazy for all consumer types:
>  >  1. spread x509 related load out across time. systems check in every 4
>  >     hours or whatever, not all at once.
>  >  2. with multiple certificate formats, there's no need to know what
>  >     version the consumer wants before they ask for it. we simply
>  >     regenerate what they ask for (via api or looking at a header on the
>  >     request or whatever).
>  >  3. fewer special cases make for cleaner code
>  >  4. if we want to be nice to distributors and let them use certs that
>  >     aren't exactly current, shouldn't we extend the same courtesy to system
>  >     consumers?
>  >
>  
>  +1 to all.
>  
>  >Doing a lazy regen implies that the real work of deciding if a
>  >certificate is still good is when we put that old certificate on the CRL
>  >list. We'd only add a certificate to the CRL immediately if it has
>  >recieved a detectable net loss in access to content, like:
>  >- quantity of entitlement has decreased (not sure if this is a real
>  >   thing or not)
>  >- content set is removed from a product (not detectable?)
>  >- content set url is changed on a product (not detectable?)
>  >- product is removed from a subscription
>  >
>  >If we hit one of those cases on a refreshpools, we add the cert cerial
>  >to the CRL, and mark it as dirty, letting the client trigger the regen
>  >at some later point.
>  
>  Quantity decreasing should probably do as it does now and go ahead
>  and revoke, this isn't really a regeneration but an outright
>  revocation, and probably is not important for the use case we're
>  trying to solve. I vote we let this immediately go to CRL and revoke
>  the cert as we do today. (it's probably a pretty rare situation
>  anyhow)
>   
I mean the quantity in the entitlement decreases, rather than the
quantity in the pool.

>  Content set changes are indeed not detectable, but possibly being
>  triggered by IT to mass regenerate via that product specific API
>  call. We will not know why we're being triggered though so as it is
>  today, we cannot decide if we're going to CRL now or later when we
>  regenerate. However as per next point I am hoping we can keep it
>  simple and have only two situations.
>  
>  For products being removed I will wager we still do not want to add
>  to the CRL immediately, driver here was to not invalidate
>  distributor certificates and my guess is we should not be doing that
>  even if we're removing access to a particular repo from their
>  subscription. Can we get clarification on this? (should we
>  immediately invalidate certificates if a product is removed from a
>  subscription)
>   
I'm pretty sure this is the exact situation where we do want to add to
the CRL. removing a product or a content set means you should no longer
have access to that content. If all a nefarious consumer has to do to
keep accessing content they're not rightly entitled to is to not ask for
a new cert, candlepin isn't doing its job.

>  Hopefully we can just stay with these two situations:
>  
>  1. we're either adding to CRL + revoking (overconsumption, normal
>  unbind, no work required because this is how it is today)
>  
>  2. we're flagging entitlement dirty now, and adding to CRL +
>  regenerating on next check-in or manifest generation
>  
>  >
>  >If we don't hit one of those cases, like if we've added a new product,
>  >we simply mark the cert as dirty. That dirty cert is still valid until
>  >the new  regen is done, but why not just keep it valid for as long as
>  >the newly regenerated version? that is, can we reuse the cert serial on
>  >the new cert, keeping the old version still valid, and reducing both the
>  >clutter in our CRL and the rate at which we consume cert serials? I
>  >think that would work.
>  
>  Having multiple certificates out in the wild with different contents
>  but the same serial, all still valid, seems strange and like a
>  probable source of future confusion, and perhaps an issue for our
>  database. Each unique cert seems like it should probably have it's
>  own serial to me. 
As far as the database goes, i think we'd be ok with just storing the
newest copy under that serial, I guess I'm ok with creating the new cert
with a new Id and revoking the old at the same time, but it would be
cool to cut down on additions to the CRL.

>  
>  If CRL clutter is a current known problem then it's worth
>  considering, but otherwise I'd propose we leave that for another day
>  and solve it if the issue arises.
>   
It's not a problem, but it is a concern. Every item we add to that list
is more memory some hash table on the cdn takes up. I'd hate to see that
thing start swapping because the file we build it too large.

>  Cheers,
>  
>  Devan 
-James

[Devan Goodwin]

On 05/25/2012 04:02 PM, James Bowes wrote:
>  I mean the quantity in the entitlement decreases, rather than the
>  quantity in the pool. 
Ah yes, that might need more fleshing out, I think I missed that there 
is a path to regenerate, not just revoke in there. I will revisit that.

> > Content set changes are indeed not detectable, but possibly being
> > triggered by IT to mass regenerate via that product specific API
> > call. We will not know why we're being triggered though so as it is
> > today, we cannot decide if we're going to CRL now or later when we
> > regenerate. However as per next point I am hoping we can keep it
> > simple and have only two situations.
> >
> > For products being removed I will wager we still do not want to add
> > to the CRL immediately, driver here was to not invalidate
> > distributor certificates and my guess is we should not be doing that
> > even if we're removing access to a particular repo from their
> > subscription. Can we get clarification on this? (should we
> > immediately invalidate certificates if a product is removed from a
> > subscription)
> >
>
>  I'm pretty sure this is the exact situation where we do want to add to
>  the CRL. removing a product or a content set means you should no longer
>  have access to that content. If all a nefarious consumer has to do to
>  keep accessing content they're not rightly entitled to is to not ask for
>  a new cert, candlepin isn't doing its job. 
It's possible but I think we need to check, if I am reading the user 
story notes correctly, removal of content sets is listed explicitly as 
one of the scenarios where we want lazy revocation. i.e. we removed some 
content from your entitlement (in this case a provided product and it's 
content), but we do not want to immediately break all those downstream 
consumers.

Candlepin could still be doing it's job if we reject immediately on loss 
of entitlement, but take a tolerant approach to rejecting certificates 
because we changed something on our end. Highly unlikely a nefarious 
consumer is going to anticipate a change to our product data, and shut 
off all consumer check-ins as far as I can forsee. It all depends on why 
this is being requested though.

[Devan Goodwin]

On Fri 25 May 2012 04:18:02 PM ADT, Devan Goodwin wrote:
>  On 05/25/2012 04:02 PM, James Bowes wrote:
> > I mean the quantity in the entitlement decreases, rather than the
> > quantity in the pool.
>
>  Ah yes, that might need more fleshing out, I think I missed that there
>  is a path to regenerate, not just revoke in there. I will revisit that. 
Ok as per IRC discussion, if a cert needs to regen because it's quantity 
changed (due to a decrease in the subscription), we will immediately 
revoke and regen.

> >
> >> Content set changes are indeed not detectable, but possibly being
> >> triggered by IT to mass regenerate via that product specific API
> >> call. We will not know why we're being triggered though so as it is
> >> today, we cannot decide if we're going to CRL now or later when we
> >> regenerate. However as per next point I am hoping we can keep it
> >> simple and have only two situations.
> >>
> >> For products being removed I will wager we still do not want to add
> >> to the CRL immediately, driver here was to not invalidate
> >> distributor certificates and my guess is we should not be doing that
> >> even if we're removing access to a particular repo from their
> >> subscription. Can we get clarification on this? (should we
> >> immediately invalidate certificates if a product is removed from a
> >> subscription)
> >>
> >
> > I'm pretty sure this is the exact situation where we do want to add to
> > the CRL. removing a product or a content set means you should no longer
> > have access to that content. If all a nefarious consumer has to do to
> > keep accessing content they're not rightly entitled to is to not ask for
> > a new cert, candlepin isn't doing its job.
>
>  It's possible but I think we need to check, if I am reading the user
>  story notes correctly, removal of content sets is listed explicitly as
>  one of the scenarios where we want lazy revocation. i.e. we removed
>  some content from your entitlement (in this case a provided product
>  and it's content), but we do not want to immediately break all those
>  downstream consumers.
>
>  Candlepin could still be doing it's job if we reject immediately on
>  loss of entitlement, but take a tolerant approach to rejecting
>  certificates because we changed something on our end. Highly unlikely
>  a nefarious consumer is going to anticipate a change to our product
>  data, and shut off all consumer check-ins as far as I can forsee. It
>  all depends on why this is being requested though.
>  
And for this we're going to assume content / product removal is still a 
lazy operation by default, but both refresh pools for owner, and 
regenerate all certs for a given product ID API calls will take an 
option to force immediate certificate revocation. This is a safety net 
in case something was granted that really should not have been.


I will write this all up on wiki and break down into tasks next week.

Cheers,

Devan

[Devan Goodwin]

On 05/25/2012 04:38 PM, Devan Goodwin wrote:
>  On Fri 25 May 2012 04:18:02 PM ADT, Devan Goodwin wrote:
> > On 05/25/2012 04:02 PM, James Bowes wrote:
> >> I mean the quantity in the entitlement decreases, rather than the
> >> quantity in the pool.
> >
> > Ah yes, that might need more fleshing out, I think I missed that there
> > is a path to regenerate, not just revoke in there. I will revisit that.
>
>  Ok as per IRC discussion, if a cert needs to regen because it's quantity
>  changed (due to a decrease in the subscription), we will immediately
>  revoke and regen.
>
> >
> >>
> >>> Content set changes are indeed not detectable, but possibly being
> >>> triggered by IT to mass regenerate via that product specific API
> >>> call. We will not know why we're being triggered though so as it is
> >>> today, we cannot decide if we're going to CRL now or later when we
> >>> regenerate. However as per next point I am hoping we can keep it
> >>> simple and have only two situations.
> >>>
> >>> For products being removed I will wager we still do not want to add
> >>> to the CRL immediately, driver here was to not invalidate
> >>> distributor certificates and my guess is we should not be doing that
> >>> even if we're removing access to a particular repo from their
> >>> subscription. Can we get clarification on this? (should we
> >>> immediately invalidate certificates if a product is removed from a
> >>> subscription)
> >>>
> >>
> >> I'm pretty sure this is the exact situation where we do want to add to
> >> the CRL. removing a product or a content set means you should no longer
> >> have access to that content. If all a nefarious consumer has to do to
> >> keep accessing content they're not rightly entitled to is to not ask for
> >> a new cert, candlepin isn't doing its job.
> >
> > It's possible but I think we need to check, if I am reading the user
> > story notes correctly, removal of content sets is listed explicitly as
> > one of the scenarios where we want lazy revocation. i.e. we removed
> > some content from your entitlement (in this case a provided product
> > and it's content), but we do not want to immediately break all those
> > downstream consumers.
> >
> > Candlepin could still be doing it's job if we reject immediately on
> > loss of entitlement, but take a tolerant approach to rejecting
> > certificates because we changed something on our end. Highly unlikely
> > a nefarious consumer is going to anticipate a change to our product
> > data, and shut off all consumer check-ins as far as I can forsee. It
> > all depends on why this is being requested though.
> >
>
>  And for this we're going to assume content / product removal is still a
>  lazy operation by default, but both refresh pools for owner, and
>  regenerate all certs for a given product ID API calls will take an
>  option to force immediate certificate revocation. This is a safety net
>  in case something was granted that really should not have been.
>
>
>  I will write this all up on wiki and break down into tasks next week.
>
>  Cheers,
>
>  Devan
>  _______________________________________________
>  candlepin mailing list
>  candlepin(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/candlepin 
Design is up at https://fedorahosted.org/candlepin/wiki/LazyCertRegen

Please have a look at let me know if anything needs to be clarified.

Thanks,

[James Bowes]

On Sat, Jun 01, 2013 at 02:37:19PM -0300, Devan Goodwin wrote:
>  On 05/25/2012 04:38 PM, Devan Goodwin wrote:
>  >On Fri 25 May 2012 04:18:02 PM ADT, Devan Goodwin wrote:
>  >>On 05/25/2012 04:02 PM, James Bowes wrote:
>  >>>I mean the quantity in the entitlement decreases, rather than the
>  >>>quantity in the pool.
>  >>
>  >>Ah yes, that might need more fleshing out, I think I missed that there
>  >>is a path to regenerate, not just revoke in there. I will revisit that.
>  >
>  >Ok as per IRC discussion, if a cert needs to regen because it's quantity
>  >changed (due to a decrease in the subscription), we will immediately
>  >revoke and regen.
>  >
>  >>
>  >>>
>  >>>>Content set changes are indeed not detectable, but possibly being
>  >>>>triggered by IT to mass regenerate via that product specific API
>  >>>>call. We will not know why we're being triggered though so as it is
>  >>>>today, we cannot decide if we're going to CRL now or later when we
>  >>>>regenerate. However as per next point I am hoping we can keep it
>  >>>>simple and have only two situations.
>  >>>>
>  >>>>For products being removed I will wager we still do not want to add
>  >>>>to the CRL immediately, driver here was to not invalidate
>  >>>>distributor certificates and my guess is we should not be doing that
>  >>>>even if we're removing access to a particular repo from their
>  >>>>subscription. Can we get clarification on this? (should we
>  >>>>immediately invalidate certificates if a product is removed from a
>  >>>>subscription)
>  >>>>
>  >>>
>  >>>I'm pretty sure this is the exact situation where we do want to add to
>  >>>the CRL. removing a product or a content set means you should no longer
>  >>>have access to that content. If all a nefarious consumer has to do to
>  >>>keep accessing content they're not rightly entitled to is to not ask for
>  >>>a new cert, candlepin isn't doing its job.
>  >>
>  >>It's possible but I think we need to check, if I am reading the user
>  >>story notes correctly, removal of content sets is listed explicitly as
>  >>one of the scenarios where we want lazy revocation. i.e. we removed
>  >>some content from your entitlement (in this case a provided product
>  >>and it's content), but we do not want to immediately break all those
>  >>downstream consumers.
>  >>
>  >>Candlepin could still be doing it's job if we reject immediately on
>  >>loss of entitlement, but take a tolerant approach to rejecting
>  >>certificates because we changed something on our end. Highly unlikely
>  >>a nefarious consumer is going to anticipate a change to our product
>  >>data, and shut off all consumer check-ins as far as I can forsee. It
>  >>all depends on why this is being requested though.
>  >>
>  >
>  >And for this we're going to assume content / product removal is still a
>  >lazy operation by default, but both refresh pools for owner, and
>  >regenerate all certs for a given product ID API calls will take an
>  >option to force immediate certificate revocation. This is a safety net
>  >in case something was granted that really should not have been.
>  >
>  >
>  >I will write this all up on wiki and break down into tasks next week.
>  >
>  >Cheers,
>  >
>  >Devan
>  >_______________________________________________
>  >candlepin mailing list
>  >candlepin(a)lists.fedorahosted.org
>  >https://fedorahosted.org/mailman/listinfo/candlepin
>  
>  Design is up at https://fedorahosted.org/candlepin/wiki/LazyCertRegen
>  
>  Please have a look at let me know if anything needs to be clarified.
>  
>  Thanks, 
under task 2 you have "Be sure". Sentance fragment or way of life?
Maybe add a note that the regeneration that goes with an immediate CRL
can be either lazy or immediate, depending on which is cleaner in the
code.

-James

[Devan Goodwin]

On 05/29/2012 02:53 PM, James Bowes wrote:
>  On Sat, Jun 01, 2013 at 02:37:19PM -0300, Devan Goodwin wrote:
> > On 05/25/2012 04:38 PM, Devan Goodwin wrote:
> >> On Fri 25 May 2012 04:18:02 PM ADT, Devan Goodwin wrote:
> >>> On 05/25/2012 04:02 PM, James Bowes wrote:
> >>>> I mean the quantity in the entitlement decreases, rather than the
> >>>> quantity in the pool.
> >>>
> >>> Ah yes, that might need more fleshing out, I think I missed that there
> >>> is a path to regenerate, not just revoke in there. I will revisit that.
> >>
> >> Ok as per IRC discussion, if a cert needs to regen because it's quantity
> >> changed (due to a decrease in the subscription), we will immediately
> >> revoke and regen.
> >>
> >>>
> >>>>
> >>>>> Content set changes are indeed not detectable, but possibly being
> >>>>> triggered by IT to mass regenerate via that product specific API
> >>>>> call. We will not know why we're being triggered though so as it
> is
> >>>>> today, we cannot decide if we're going to CRL now or later when
> we
> >>>>> regenerate. However as per next point I am hoping we can keep it
> >>>>> simple and have only two situations.
> >>>>>
> >>>>> For products being removed I will wager we still do not want to add
> >>>>> to the CRL immediately, driver here was to not invalidate
> >>>>> distributor certificates and my guess is we should not be doing that
> >>>>> even if we're removing access to a particular repo from their
> >>>>> subscription. Can we get clarification on this? (should we
> >>>>> immediately invalidate certificates if a product is removed from a
> >>>>> subscription)
> >>>>>
> >>>>
> >>>> I'm pretty sure this is the exact situation where we do want to add
> to
> >>>> the CRL. removing a product or a content set means you should no longer
> >>>> have access to that content. If all a nefarious consumer has to do to
> >>>> keep accessing content they're not rightly entitled to is to not ask
> for
> >>>> a new cert, candlepin isn't doing its job.
> >>>
> >>> It's possible but I think we need to check, if I am reading the user
> >>> story notes correctly, removal of content sets is listed explicitly as
> >>> one of the scenarios where we want lazy revocation. i.e. we removed
> >>> some content from your entitlement (in this case a provided product
> >>> and it's content), but we do not want to immediately break all those
> >>> downstream consumers.
> >>>
> >>> Candlepin could still be doing it's job if we reject immediately on
> >>> loss of entitlement, but take a tolerant approach to rejecting
> >>> certificates because we changed something on our end. Highly unlikely
> >>> a nefarious consumer is going to anticipate a change to our product
> >>> data, and shut off all consumer check-ins as far as I can forsee. It
> >>> all depends on why this is being requested though.
> >>>
> >>
> >> And for this we're going to assume content / product removal is still a
> >> lazy operation by default, but both refresh pools for owner, and
> >> regenerate all certs for a given product ID API calls will take an
> >> option to force immediate certificate revocation. This is a safety net
> >> in case something was granted that really should not have been.
> >>
> >>
> >> I will write this all up on wiki and break down into tasks next week.
> >>
> >> Cheers,
> >>
> >> Devan
> >> _______________________________________________
> >> candlepin mailing list
> >> candlepin(a)lists.fedorahosted.org
> >> https://fedorahosted.org/mailman/listinfo/candlepin
> >
> > Design is up at https://fedorahosted.org/candlepin/wiki/LazyCertRegen
> >
> > Please have a look at let me know if anything needs to be clarified.
> >
> > Thanks,
>
>  under task 2 you have "Be sure". Sentance fragment or way of life?
>  Maybe add a note that the regeneration that goes with an immediate CRL
>  can be either lazy or immediate, depending on which is cleaner in the
>  code.
>
>  -James 
Thanks, added.

Sorry about the previous future email as well all, date testing cert 
expiration again. :)

Devan

[Devan Goodwin]

On 05/29/2012 02:53 PM, James Bowes wrote:
>  On Sat, Jun 01, 2013 at 02:37:19PM -0300, Devan Goodwin wrote:
> > On 05/25/2012 04:38 PM, Devan Goodwin wrote:
> >> On Fri 25 May 2012 04:18:02 PM ADT, Devan Goodwin wrote:
> >>> On 05/25/2012 04:02 PM, James Bowes wrote:
> >>>> I mean the quantity in the entitlement decreases, rather than the
> >>>> quantity in the pool.
> >>>
> >>> Ah yes, that might need more fleshing out, I think I missed that there
> >>> is a path to regenerate, not just revoke in there. I will revisit that.
> >>
> >> Ok as per IRC discussion, if a cert needs to regen because it's quantity
> >> changed (due to a decrease in the subscription), we will immediately
> >> revoke and regen.
> >>
> >>>
> >>>>
> >>>>> Content set changes are indeed not detectable, but possibly being
> >>>>> triggered by IT to mass regenerate via that product specific API
> >>>>> call. We will not know why we're being triggered though so as it
> is
> >>>>> today, we cannot decide if we're going to CRL now or later when
> we
> >>>>> regenerate. However as per next point I am hoping we can keep it
> >>>>> simple and have only two situations.
> >>>>>
> >>>>> For products being removed I will wager we still do not want to add
> >>>>> to the CRL immediately, driver here was to not invalidate
> >>>>> distributor certificates and my guess is we should not be doing that
> >>>>> even if we're removing access to a particular repo from their
> >>>>> subscription. Can we get clarification on this? (should we
> >>>>> immediately invalidate certificates if a product is removed from a
> >>>>> subscription)
> >>>>>
> >>>>
> >>>> I'm pretty sure this is the exact situation where we do want to add
> to
> >>>> the CRL. removing a product or a content set means you should no longer
> >>>> have access to that content. If all a nefarious consumer has to do to
> >>>> keep accessing content they're not rightly entitled to is to not ask
> for
> >>>> a new cert, candlepin isn't doing its job.
> >>>
> >>> It's possible but I think we need to check, if I am reading the user
> >>> story notes correctly, removal of content sets is listed explicitly as
> >>> one of the scenarios where we want lazy revocation. i.e. we removed
> >>> some content from your entitlement (in this case a provided product
> >>> and it's content), but we do not want to immediately break all those
> >>> downstream consumers.
> >>>
> >>> Candlepin could still be doing it's job if we reject immediately on
> >>> loss of entitlement, but take a tolerant approach to rejecting
> >>> certificates because we changed something on our end. Highly unlikely
> >>> a nefarious consumer is going to anticipate a change to our product
> >>> data, and shut off all consumer check-ins as far as I can forsee. It
> >>> all depends on why this is being requested though.
> >>>
> >>
> >> And for this we're going to assume content / product removal is still a
> >> lazy operation by default, but both refresh pools for owner, and
> >> regenerate all certs for a given product ID API calls will take an
> >> option to force immediate certificate revocation. This is a safety net
> >> in case something was granted that really should not have been.
> >>
> >>
> >> I will write this all up on wiki and break down into tasks next week.
> >>
> >> Cheers,
> >>
> >> Devan
> >> _______________________________________________
> >> candlepin mailing list
> >> candlepin(a)lists.fedorahosted.org
> >> https://fedorahosted.org/mailman/listinfo/candlepin
> >
> > Design is up at https://fedorahosted.org/candlepin/wiki/LazyCertRegen
> >
> > Please have a look at let me know if anything needs to be clarified.
> >
> > Thanks,
>
>  under task 2 you have "Be sure". Sentance fragment or way of life?
>  Maybe add a note that the regeneration that goes with an immediate CRL
>  can be either lazy or immediate, depending on which is cleaner in the
>  code.
>
>  -James 
Thanks, added.

Sorry about the previous future email as well all, date testing cert 
expiration again. :)

Devan

[Bryan Kearney]

On 05/29/2012 02:16 PM, Devan Goodwin wrote:
>  On 05/29/2012 02:53 PM, James Bowes wrote:
> > On Sat, Jun 01, 2013 at 02:37:19PM -0300, Devan Goodwin wrote:
> >> On 05/25/2012 04:38 PM, Devan Goodwin wrote:
> >>> On Fri 25 May 2012 04:18:02 PM ADT, Devan Goodwin wrote:
> >>>> On 05/25/2012 04:02 PM, James Bowes wrote:
> >>>>> I mean the quantity in the entitlement decreases, rather than the
> >>>>> quantity in the pool.
> >>>>
> >>>> Ah yes, that might need more fleshing out, I think I missed that there
> >>>> is a path to regenerate, not just revoke in there. I will revisit
> >>>> that.
> >>>
> >>> Ok as per IRC discussion, if a cert needs to regen because it's
> >>> quantity
> >>> changed (due to a decrease in the subscription), we will immediately
> >>> revoke and regen.
> >>>
> >>>>
> >>>>>
> >>>>>> Content set changes are indeed not detectable, but possibly
> being
> >>>>>> triggered by IT to mass regenerate via that product specific API
> >>>>>> call. We will not know why we're being triggered though so as
> it is
> >>>>>> today, we cannot decide if we're going to CRL now or later
> when we
> >>>>>> regenerate. However as per next point I am hoping we can keep it
> >>>>>> simple and have only two situations.
> >>>>>>
> >>>>>> For products being removed I will wager we still do not want to
> add
> >>>>>> to the CRL immediately, driver here was to not invalidate
> >>>>>> distributor certificates and my guess is we should not be doing
> that
> >>>>>> even if we're removing access to a particular repo from
> their
> >>>>>> subscription. Can we get clarification on this? (should we
> >>>>>> immediately invalidate certificates if a product is removed from
> a
> >>>>>> subscription)
> >>>>>>
> >>>>>
> >>>>> I'm pretty sure this is the exact situation where we do want to
> >>>>> add to
> >>>>> the CRL. removing a product or a content set means you should no
> >>>>> longer
> >>>>> have access to that content. If all a nefarious consumer has to do
> to
> >>>>> keep accessing content they're not rightly entitled to is to not
> >>>>> ask for
> >>>>> a new cert, candlepin isn't doing its job.
> >>>>
> >>>> It's possible but I think we need to check, if I am reading the user
> >>>> story notes correctly, removal of content sets is listed explicitly as
> >>>> one of the scenarios where we want lazy revocation. i.e. we removed
> >>>> some content from your entitlement (in this case a provided product
> >>>> and it's content), but we do not want to immediately break all those
> >>>> downstream consumers.
> >>>>
> >>>> Candlepin could still be doing it's job if we reject immediately on
> >>>> loss of entitlement, but take a tolerant approach to rejecting
> >>>> certificates because we changed something on our end. Highly unlikely
> >>>> a nefarious consumer is going to anticipate a change to our product
> >>>> data, and shut off all consumer check-ins as far as I can forsee. It
> >>>> all depends on why this is being requested though.
> >>>>
> >>>
> >>> And for this we're going to assume content / product removal is still a
> >>> lazy operation by default, but both refresh pools for owner, and
> >>> regenerate all certs for a given product ID API calls will take an
> >>> option to force immediate certificate revocation. This is a safety net
> >>> in case something was granted that really should not have been.
> >>>
> >>>
> >>> I will write this all up on wiki and break down into tasks next week.
> >>>
> >>> Cheers,
> >>>
> >>> Devan
> >>> _______________________________________________
> >>> candlepin mailing list
> >>> candlepin(a)lists.fedorahosted.org
> >>> https://fedorahosted.org/mailman/listinfo/candlepin
> >>
> >> Design is up at https://fedorahosted.org/candlepin/wiki/LazyCertRegen
> >>
> >> Please have a look at let me know if anything needs to be clarified.
> >>
> >> Thanks,
> >
> > under task 2 you have "Be sure". Sentance fragment or way of life?
> > Maybe add a note that the regeneration that goes with an immediate CRL
> > can be either lazy or immediate, depending on which is cleaner in the
> > code.
> >
> > -James
>  
Why get consumers/{uid}? That seems like an odd side effect?

[Devan Goodwin]

On 05/29/2012 03:40 PM, Bryan Kearney wrote:
>  On 05/29/2012 02:16 PM, Devan Goodwin wrote:
> > On 05/29/2012 02:53 PM, James Bowes wrote:
> >> On Sat, Jun 01, 2013 at 02:37:19PM -0300, Devan Goodwin wrote:
> >>> On 05/25/2012 04:38 PM, Devan Goodwin wrote:
> >>>> On Fri 25 May 2012 04:18:02 PM ADT, Devan Goodwin wrote:
> >>>>> On 05/25/2012 04:02 PM, James Bowes wrote:
> >>>>>> I mean the quantity in the entitlement decreases, rather than
> the
> >>>>>> quantity in the pool.
> >>>>>
> >>>>> Ah yes, that might need more fleshing out, I think I missed that
> >>>>> there
> >>>>> is a path to regenerate, not just revoke in there. I will revisit
> >>>>> that.
> >>>>
> >>>> Ok as per IRC discussion, if a cert needs to regen because it's
> >>>> quantity
> >>>> changed (due to a decrease in the subscription), we will immediately
> >>>> revoke and regen.
> >>>>
> >>>>>
> >>>>>>
> >>>>>>> Content set changes are indeed not detectable, but possibly
> being
> >>>>>>> triggered by IT to mass regenerate via that product specific
> API
> >>>>>>> call. We will not know why we're being triggered though
> so as it is
> >>>>>>> today, we cannot decide if we're going to CRL now or
> later when we
> >>>>>>> regenerate. However as per next point I am hoping we can keep
> it
> >>>>>>> simple and have only two situations.
> >>>>>>>
> >>>>>>> For products being removed I will wager we still do not want
> to add
> >>>>>>> to the CRL immediately, driver here was to not invalidate
> >>>>>>> distributor certificates and my guess is we should not be
> doing
> >>>>>>> that
> >>>>>>> even if we're removing access to a particular repo from
> their
> >>>>>>> subscription. Can we get clarification on this? (should we
> >>>>>>> immediately invalidate certificates if a product is removed
> from a
> >>>>>>> subscription)
> >>>>>>>
> >>>>>>
> >>>>>> I'm pretty sure this is the exact situation where we do want
> to
> >>>>>> add to
> >>>>>> the CRL. removing a product or a content set means you should no
> >>>>>> longer
> >>>>>> have access to that content. If all a nefarious consumer has to
> >>>>>> do to
> >>>>>> keep accessing content they're not rightly entitled to is to
> not
> >>>>>> ask for
> >>>>>> a new cert, candlepin isn't doing its job.
> >>>>>
> >>>>> It's possible but I think we need to check, if I am reading the
> user
> >>>>> story notes correctly, removal of content sets is listed
> >>>>> explicitly as
> >>>>> one of the scenarios where we want lazy revocation. i.e. we removed
> >>>>> some content from your entitlement (in this case a provided product
> >>>>> and it's content), but we do not want to immediately break all
> those
> >>>>> downstream consumers.
> >>>>>
> >>>>> Candlepin could still be doing it's job if we reject immediately
> on
> >>>>> loss of entitlement, but take a tolerant approach to rejecting
> >>>>> certificates because we changed something on our end. Highly
> unlikely
> >>>>> a nefarious consumer is going to anticipate a change to our product
> >>>>> data, and shut off all consumer check-ins as far as I can forsee. It
> >>>>> all depends on why this is being requested though.
> >>>>>
> >>>>
> >>>> And for this we're going to assume content / product removal is
> >>>> still a
> >>>> lazy operation by default, but both refresh pools for owner, and
> >>>> regenerate all certs for a given product ID API calls will take an
> >>>> option to force immediate certificate revocation. This is a safety net
> >>>> in case something was granted that really should not have been.
> >>>>
> >>>>
> >>>> I will write this all up on wiki and break down into tasks next week.
> >>>>
> >>>> Cheers,
> >>>>
> >>>> Devan
> >>>> _______________________________________________
> >>>> candlepin mailing list
> >>>> candlepin(a)lists.fedorahosted.org
> >>>> https://fedorahosted.org/mailman/listinfo/candlepin
> >>>
> >>> Design is up at https://fedorahosted.org/candlepin/wiki/LazyCertRegen
> >>>
> >>> Please have a look at let me know if anything needs to be clarified.
> >>>
> >>> Thanks,
> >>
> >> under task 2 you have "Be sure". Sentance fragment or way of life?
> >> Maybe add a note that the regeneration that goes with an immediate CRL
> >> can be either lazy or immediate, depending on which is cleaner in the
> >> code.
> >>
> >> -James
> >
>
>  Why get consumers/{uid}? That seems like an odd side effect?
>  _______________________________________________
>  candlepin mailing list
>  candlepin(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/candlepin 
My mistake, I thought entitlement certificates were in that JSON but 
they are not. I will update design and remove it.

[James Bowes]

On Fri, May 25, 2012 at 04:02:40PM -0300, James Bowes wrote:
<snip>
>  > >If we don't hit one of those cases, like if we've
> added a new product,
>  > >we simply mark the cert as dirty. That dirty cert is still valid until
>  > >the new  regen is done, but why not just keep it valid for as long as
>  > >the newly regenerated version? that is, can we reuse the cert serial on
>  > >the new cert, keeping the old version still valid, and reducing both the
>  > >clutter in our CRL and the rate at which we consume cert serials? I
>  > >think that would work.
>  > 
>  > Having multiple certificates out in the wild with different contents
>  > but the same serial, all still valid, seems strange and like a
>  > probable source of future confusion, and perhaps an issue for our
>  > database. Each unique cert seems like it should probably have it's
>  > own serial to me.
>  
>  As far as the database goes, i think we'd be ok with just storing the
>  newest copy under that serial, I guess I'm ok with creating the new cert
>  with a new Id and revoking the old at the same time, but it would be
>  cool to cut down on additions to the CRL.
>   
Forget this part. If we keep the same serial, that will most definately
break clients! silly me.

>  > 
>  > If CRL clutter is a current known problem then it's worth
>  > considering, but otherwise I'd propose we leave that for another day
>  > and solve it if the issue arises.
>  > 
>  
>  It's not a problem, but it is a concern. Every item we add to that list
>  is more memory some hash table on the cdn takes up. I'd hate to see that
>  thing start swapping because the file we build it too large.
>  
>  > Cheers,
>  > 
>  > Devan
>  
>  -James 


>  _______________________________________________
>  candlepin mailing list
>  candlepin(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/candlepin 

-James