I built my first 5.6 custom kernel from the src.rpm yesterday in F31. And my patch to enable the use of a daemon I run to gather entropy from an rtl2832 (atmospheric) and put it into the kernel to keep the entropy pool full failed. This has happened in the past, that's why I have to patch, but the interface was never removed before. If it has been removed, can you point me to the discussion that led to that decision. I have to determine how secure it will be to modify random.c again in order to continue feeding entropy to the kernel.
On another note, an observation. There seem to be pretty frequent major revisions in the kernel PRNG generator. In some ways this is a good thing, because it indicates that people are paying attention to it. But it also means that past versions have been deemed unsuitable, which decreases confidence that the current version is suitable. So, I would be appreciative if you could also point me to the discussion surrounding the latest major change.
Thanks.
* stan:
I built my first 5.6 custom kernel from the src.rpm yesterday in F31. And my patch to enable the use of a daemon I run to gather entropy from an rtl2832 (atmospheric) and put it into the kernel to keep the entropy pool full failed. This has happened in the past, that's why I have to patch, but the interface was never removed before. If it has been removed, can you point me to the discussion that led to that decision.
How does the removal of the interface materialize itself?
Thanks, Florian
On Mon, 24 Feb 2020 17:29:07 +0100 Florian Weimer fweimer@redhat.com wrote:
- stan:
I built my first 5.6 custom kernel from the src.rpm yesterday in F31. And my patch to enable the use of a daemon I run to gather entropy from an rtl2832 (atmospheric) and put it into the kernel to keep the entropy pool full failed. This has happened in the past, that's why I have to patch, but the interface was never removed before. If it has been removed, can you point me to the discussion that led to that decision.
I haven't done a complete analysis yet, the changes are pretty extensive. But the marker that the callback used to trigger the daemon has been removed.
- .procname = "read_wakeup_threshold", - .data = &random_read_wakeup_bits, - .maxlen = sizeof(int), - .mode = 0644, - .proc_handler = proc_dointvec_minmax, - .extra1 = &min_read_thresh, - .extra2 = &max_read_thresh, - },
This seems to have been replaced with hard-coded functions that read from specific sources (mouse, key strokes, hard drives, etc.) to gather system entropy.
I wanted to see the rationale for the changes before I invested the time to see how it is all working together now, and how to insert my code without disrupting everything. This is a pretty critical part of the kernel (I would say vital), so I like to be sure that everything is making sense, and that it was vetted properly.
I'm not an expert in this, so I could be reading it all wrong, but I want to investigate before I decide. The developer description of the changes and the reasoning behind them would be the place to start. Maybe the decision was that no one was using this interface, so it didn't make sense to keep it around (more code to rot, and threaten security).
On 2/24/20 1:24 PM, stan wrote:
On Mon, 24 Feb 2020 17:29:07 +0100 Florian Weimer fweimer@redhat.com wrote:
- stan:
I built my first 5.6 custom kernel from the src.rpm yesterday in F31. And my patch to enable the use of a daemon I run to gather entropy from an rtl2832 (atmospheric) and put it into the kernel to keep the entropy pool full failed. This has happened in the past, that's why I have to patch, but the interface was never removed before. If it has been removed, can you point me to the discussion that led to that decision.
I haven't done a complete analysis yet, the changes are pretty extensive. But the marker that the callback used to trigger the daemon has been removed.
.procname = "read_wakeup_threshold",.data = &random_read_wakeup_bits,.maxlen = sizeof(int),.mode = 0644,.proc_handler = proc_dointvec_minmax,.extra1 = &min_read_thresh,.extra2 = &max_read_thresh,},This seems to have been replaced with hard-coded functions that read from specific sources (mouse, key strokes, hard drives, etc.) to gather system entropy.
I wanted to see the rationale for the changes before I invested the time to see how it is all working together now, and how to insert my code without disrupting everything. This is a pretty critical part of the kernel (I would say vital), so I like to be sure that everything is making sense, and that it was vetted properly.
I'm not an expert in this, so I could be reading it all wrong, but I want to investigate before I decide. The developer description of the changes and the reasoning behind them would be the place to start. Maybe the decision was that no one was using this interface, so it didn't make sense to keep it around (more code to rot, and threaten security).
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/dr...
And the thread if you'd like to read https://lore.kernel.org/linux-api/cover.1577088521.git.luto@kernel.org/
On Mon, 24 Feb 2020 14:40:12 -0500 Laura Abbott laura@labbott.name wrote:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/dr...
And the thread if you'd like to read https://lore.kernel.org/linux-api/cover.1577088521.git.luto@kernel.org/
Thank you Laura! That is just what I was looking for. Though not what I was hoping for. :-)
They've just decided to make the kernel robust to the situation found in server farms. So, I can patch back to what I need to feed entropy into the kernel with no security concerns. Actually, it will be more secure since the kernel in a home system uses no where near the output of the rtl2832 (~ 90 KBytes / sec). Not enough for monte carlo, but plenty for the kernel and small simulations. The kernel entropy pool is 4096 bits, 512 bytes.
As they say, it probably isn't necessary because the PRNG is secure under most (all?) conditions, but this can be thought of as suspenders, just in case there *is* a back door in the cha cha algorithm.
On Mon, Feb 24, 2020 at 02:40:12PM -0500, Laura Abbott wrote:
On 2/24/20 1:24 PM, stan wrote:
On Mon, 24 Feb 2020 17:29:07 +0100 Florian Weimer fweimer@redhat.com wrote:
- stan:
I built my first 5.6 custom kernel from the src.rpm yesterday in F31. And my patch to enable the use of a daemon I run to gather entropy from an rtl2832 (atmospheric) and put it into the kernel to keep the entropy pool full failed. This has happened in the past, that's why I have to patch, but the interface was never removed before. If it has been removed, can you point me to the discussion that led to that decision.
Just out of curiosity, instead of patching the kernel to feed entropy from your SDN directly into the kernel, have you considered just adding this as an entropy source to rngd? That would save you the trouble of having to patch the kernel to do this. I'd be interested in getting a feature like that into place for rngd
Neil
I haven't done a complete analysis yet, the changes are pretty extensive. But the marker that the callback used to trigger the daemon has been removed.
.procname = "read_wakeup_threshold",.data = &random_read_wakeup_bits,.maxlen = sizeof(int),.mode = 0644,.proc_handler = proc_dointvec_minmax,.extra1 = &min_read_thresh,.extra2 = &max_read_thresh,},This seems to have been replaced with hard-coded functions that read from specific sources (mouse, key strokes, hard drives, etc.) to gather system entropy.
I wanted to see the rationale for the changes before I invested the time to see how it is all working together now, and how to insert my code without disrupting everything. This is a pretty critical part of the kernel (I would say vital), so I like to be sure that everything is making sense, and that it was vetted properly.
I'm not an expert in this, so I could be reading it all wrong, but I want to investigate before I decide. The developer description of the changes and the reasoning behind them would be the place to start. Maybe the decision was that no one was using this interface, so it didn't make sense to keep it around (more code to rot, and threaten security).
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/dr...
And the thread if you'd like to read https://lore.kernel.org/linux-api/cover.1577088521.git.luto@kernel.org/ _______________________________________________ kernel mailing list -- kernel@lists.fedoraproject.org To unsubscribe send an email to kernel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/kernel@lists.fedoraproject.org
On Tue, 25 Feb 2020 08:17:37 -0500 Neil Horman nhorman@redhat.com wrote:
Just out of curiosity, instead of patching the kernel to feed entropy from your SDN directly into the kernel, have you considered just adding this as an entropy source to rngd? That would save you the trouble of having to patch the kernel to do this. I'd be interested in getting a feature like that into place for rngd
I didn't. Even though I used rngtest while validating the output. Hmmm. These changes to random.c mean that rngd is also going to stop working to feed entropy as well, since /dev/random can no longer be used to feed entropy into the pool (that's how I understand the changes, at least).
That said, I have no issues with integration into rngd. The code is released gplv3 on sourceforge, but I can relicense it as gplv2 for you if needed. It hasn't changed since I developed it, only the way of getting it into the kernel and using it there has needed to be updated. A few iterations ago, random.c removed the ability to reseed the crng on a timed basis, instead checking whether there was enough entropy for a reseed and a cool down period had passed each time a call was made to get random bytes. I also re-introduced periodic reseeds; entropy to burn, why not use it. I checked the output last year, and it was still passing all the tests. I used the NSA tests this time, as well as rngtest and dieharder, and they also passed. Of course, passing tests isn't a guarantee, but it does give a level of confidence.
I modelled the daemon on the existing audio-entropyd, which I also use. It isn't very efficient, but a paper I read a few years ago found that multiple sources of uncorrelated entropy, even 'bad' entropy, meaning there might be only 2 or 3 bits of entropy per byte, made crngs more robust. Sort of like mercury concentrating in fish as it goes up the food chain. And the kernel does a hash of all the entropy it is fed, so even though that doesn't add entropy, we don't have the tools to find the weaknesses spread out over 128 or 256 bytes. At least I don't *think* there are tools capable of doing that. Maybe quantum tools will have that capability, and hashes will have to be rethought.
Here is the source for the daemon. http://rtl2832-entropyd.sourceforge.net/
If you are doing this, you might want to look at adding the bit-babbler to rngd also. I considered it, but at ~ $100 it is a little pricey and overkill for my needs. I would still have needed to find a way to enter the entropy into the kernel, so I just went with the cheaper option; an rtl2832 is about $10 on ebay, or was when I bought mine. And more than adequate.
I think that any server farm or cloud provider that isn't using an entropy device to reseed their crngs periodically is acting in a criminally negligent manner with their customers' data. Especially when it is so cheap. Even this $10 device is capable of reseeding ~100,000 servers every 10 minutes or so. Sure, there would be some one time development costs, but that's just a cost of doing business.
That's why all this hand wringing over lack of entropy seems strange to me. There are lots of adequate entropy devices, some expensive, some reasonably priced, why isn't integration for them added instead of deciding to just make do with crngs? We have network storage devices, network time devices, why not network entropy devices?
On Tue, 25 Feb 2020 09:27:07 -0700 stan upaitag@zoho.com wrote:
year, and it was still passing all the tests. I used the NSA tests
Correction: NIST, not NSA, though I am sure the NSA is aware of these tests.
On Tue, Feb 25, 2020 at 09:27:07AM -0700, stan wrote:
On Tue, 25 Feb 2020 08:17:37 -0500 Neil Horman nhorman@redhat.com wrote:
Just out of curiosity, instead of patching the kernel to feed entropy from your SDN directly into the kernel, have you considered just adding this as an entropy source to rngd? That would save you the trouble of having to patch the kernel to do this. I'd be interested in getting a feature like that into place for rngd
I didn't. Even though I used rngtest while validating the output. Hmmm. These changes to random.c mean that rngd is also going to stop working to feed entropy as well, since /dev/random can no longer be used to feed entropy into the pool (that's how I understand the changes, at least).
Thats not my understanding. As I understand the changes, /dev/random has been converted so that its no longer blocks (which is why the removed the read_wakeup_threshold, since theres never a case where /dev/random will block anymore). That doesn't prevent rngd from feeding new entropy into the kernel though, via /dev/randoms RNDADDTOENTCNT and RNDADDENTROPY ioctls (which is how we feed in more entropy)
That said, I have no issues with integration into rngd. The code is released gplv3 on sourceforge, but I can relicense it as gplv2 for you if needed. It hasn't changed since I developed it, only the way of getting it into the kernel and using it there has needed to be updated.
I'm fine with gplv3, IIRC rngd was initially licensed as gplv2 or later, so it should be good.
A few iterations ago, random.c removed the ability to reseed the crng on a timed basis, instead checking whether there was enough entropy for a reseed and a cool down period had passed each time a call was made to get random bytes. I also re-introduced periodic reseeds; entropy to burn, why not use it. I checked the output last year, and it was still passing all the tests. I used the NSA tests this time, as well as rngtest and dieharder, and they also passed. Of course, passing tests isn't a guarantee, but it does give a level of confidence.
I modelled the daemon on the existing audio-entropyd, which I also use. It isn't very efficient, but a paper I read a few years ago found that multiple sources of uncorrelated entropy, even 'bad' entropy, meaning there might be only 2 or 3 bits of entropy per byte, made crngs more robust. Sort of like mercury concentrating in fish as it goes up the food chain. And the kernel does a hash of all the entropy it is fed, so even though that doesn't add entropy, we don't have the tools to find the weaknesses spread out over 128 or 256 bytes. At least I don't *think* there are tools capable of doing that. Maybe quantum tools will have that capability, and hashes will have to be rethought.
Here is the source for the daemon. http://rtl2832-entropyd.sourceforge.net/
i'll take a look at it, thanks!
If you are doing this, you might want to look at adding the bit-babbler to rngd also. I considered it, but at ~ $100 it is a little pricey and overkill for my needs. I would still have needed to find a way to enter the entropy into the kernel, so I just went with the cheaper option; an rtl2832 is about $10 on ebay, or was when I bought mine. And more than adequate.
Yeah, I just ordered an RTL2832U from amazon for a few bucks, seems like a good cheap entropy source to make available. I'll try look into bit-babbler as well, but at $100, that might not be as worthwhile.
I think that any server farm or cloud provider that isn't using an entropy device to reseed their crngs periodically is acting in a criminally negligent manner with their customers' data. Especially when it is so cheap. Even this $10 device is capable of reseeding ~100,000 servers every 10 minutes or so. Sure, there would be some one time development costs, but that's just a cost of doing business.
That's why all this hand wringing over lack of entropy seems strange to me. There are lots of adequate entropy devices, some expensive, some reasonably priced, why isn't integration for them added instead of deciding to just make do with crngs? We have network storage devices, network time devices, why not network entropy devices?
Usually network entropy is avoided because its subject to manipulation from off system. You can hammer a target card enough that you can do enough prediction of interrupt timing to predict what the outcome will be.
As for radio sources, I'm not sure. $10 is actually a huge cost on a BOM when you're building 1000's of systems, and crngs are cheaper, especially when an OS adds them anyway to handle the 'no hardware source' use case.
Neil
On Tue, 25 Feb 2020 13:13:16 -0500 Neil Horman nhorman@redhat.com wrote:
Thats not my understanding. As I understand the changes, /dev/random has been converted so that its no longer blocks (which is why the removed the read_wakeup_threshold, since theres never a case where /dev/random will block anymore). That doesn't prevent rngd from feeding new entropy into the kernel though, via /dev/randoms RNDADDTOENTCNT and RNDADDENTROPY ioctls (which is how we feed in more entropy)
If you are right, that is excellent. I've been hesitating with creating the patch because it is becoming like recreating more and more of random.c. I have to be really careful because the kernel expects the new interface, so I have to leave it, but I still have to add the obsolete interface back for my use.
Doesn't the elimination of the shadow pool, and the removal of push_to_pool end the ability to push entropy? I'm going to have to bite the bullet and take the code apart until I can understand the new system.
I'm fine with gplv3, IIRC rngd was initially licensed as gplv2 or later, so it should be good.
Great!
Yeah, I just ordered an RTL2832U from amazon for a few bucks, seems like a good cheap entropy source to make available. I'll try look into bit-babbler as well, but at $100, that might not be as worthwhile.
Yeah, I was thinking of for server farm and cloud provider usage.
Usually network entropy is avoided because its subject to manipulation from off system. You can hammer a target card enough that you can do enough prediction of interrupt timing to predict what the outcome will be.
I'm thinking of a dedicated server that does nothing but provide entropy. A couple of different sources of entropy, and a private network address. The servers sign up as clients, and the entropy server sends them entropy updates on a periodic basis, which wakes the client and reseeds the crng.
As for radio sources, I'm not sure. $10 is actually a huge cost on a BOM when you're building 1000's of systems, and crngs are cheaper, especially when an OS adds them anyway to handle the 'no hardware source' use case.
Sure, individual entropy sources for each server is overkill and too expensive. But if it's spread across thousands of servers via a dedicated entropy server, it's just a few cents a server, or less.
On Tue, Feb 25, 2020 at 11:58:56AM -0700, stan wrote:
On Tue, 25 Feb 2020 13:13:16 -0500 Neil Horman nhorman@redhat.com wrote:
Thats not my understanding. As I understand the changes, /dev/random has been converted so that its no longer blocks (which is why the removed the read_wakeup_threshold, since theres never a case where /dev/random will block anymore). That doesn't prevent rngd from feeding new entropy into the kernel though, via /dev/randoms RNDADDTOENTCNT and RNDADDENTROPY ioctls (which is how we feed in more entropy)
If you are right, that is excellent. I've been hesitating with creating the patch because it is becoming like recreating more and more of random.c. I have to be really careful because the kernel expects the new interface, so I have to leave it, but I still have to add the obsolete interface back for my use.
Doesn't the elimination of the shadow pool, and the removal of push_to_pool end the ability to push entropy? I'm going to have to bite the bullet and take the code apart until I can understand the new system.
shouldn't do, the ioclt writes directly to the input_pool
I'm fine with gplv3, IIRC rngd was initially licensed as gplv2 or later, so it should be good.
Great!
Yeah, I just ordered an RTL2832U from amazon for a few bucks, seems like a good cheap entropy source to make available. I'll try look into bit-babbler as well, but at $100, that might not be as worthwhile.
Yeah, I was thinking of for server farm and cloud provider usage.
Usually network entropy is avoided because its subject to manipulation from off system. You can hammer a target card enough that you can do enough prediction of interrupt timing to predict what the outcome will be.
I'm thinking of a dedicated server that does nothing but provide entropy. A couple of different sources of entropy, and a private network address. The servers sign up as clients, and the entropy server sends them entropy updates on a periodic basis, which wakes the client and reseeds the crng.
That works pretty well in a secure environment (in fact, once I get it fixed, you can use the nist beacon server code and the nist beacon source to transport that entropy), but I don't think cloud providers like the idea of shipping entropy from a central source to other nodes where the possibility exists for snooping. They all rely on localized entropy. Shared entropy pools accross systems are better used for things like distributed testing, where multiple systems might need to use the same random bits.
As for radio sources, I'm not sure. $10 is actually a huge cost on a BOM when you're building 1000's of systems, and crngs are cheaper, especially when an OS adds them anyway to handle the 'no hardware source' use case.
Sure, individual entropy sources for each server is overkill and too expensive. But if it's spread across thousands of servers via a dedicated entropy server, it's just a few cents a server, or less.
On Tue, 25 Feb 2020 14:24:10 -0500 Neil Horman nhorman@redhat.com wrote:
On Tue, Feb 25, 2020 at 11:58:56AM -0700, stan wrote:
Doesn't the elimination of the shadow pool, and the removal of push_to_pool end the ability to push entropy? I'm going to have to bite the bullet and take the code apart until I can understand the new system.
shouldn't do, the ioclt writes directly to the input_pool
I can confirm this. I booted the 5.6 kernel without any patches, and the rtl2832 is happily feeding it entropy. So, all this back and forth was because of my mistaken understanding. Apologies everyone, and thanks for your patience.
I'm thinking of a dedicated server that does nothing but provide entropy.
That works pretty well in a secure environment (in fact, once I get it fixed, you can use the nist beacon server code and the nist beacon source to transport that entropy), but I don't think cloud providers like the idea of shipping entropy from a central source to other nodes where the possibility exists for snooping. They all rely on localized entropy. Shared entropy pools accross systems are better used for things like distributed testing, where multiple systems might need to use the same random bits.
Thanks for the explanation.
On Tue, Feb 25, 2020 at 12:48:08PM -0700, stan wrote:
On Tue, 25 Feb 2020 14:24:10 -0500 Neil Horman nhorman@redhat.com wrote:
On Tue, Feb 25, 2020 at 11:58:56AM -0700, stan wrote:
Doesn't the elimination of the shadow pool, and the removal of push_to_pool end the ability to push entropy? I'm going to have to bite the bullet and take the code apart until I can understand the new system.
shouldn't do, the ioclt writes directly to the input_pool
I can confirm this. I booted the 5.6 kernel without any patches, and the rtl2832 is happily feeding it entropy. So, all this back and forth was because of my mistaken understanding. Apologies everyone, and thanks for your patience.
I'm thinking of a dedicated server that does nothing but provide entropy.
That works pretty well in a secure environment (in fact, once I get it fixed, you can use the nist beacon server code and the nist beacon source to transport that entropy), but I don't think cloud providers like the idea of shipping entropy from a central source to other nodes where the possibility exists for snooping. They all rely on localized entropy. Shared entropy pools accross systems are better used for things like distributed testing, where multiple systems might need to use the same random bits.
Thanks for the explanation.
Hey, just FYI, its still a bit nascent, but it works: https://github.com/nhorman/rng-tools
Now has an rtlsdr entropy source, which can be used to feed the kernel entropy pool.
I'll be tagging and releasing an updated rng-tools in the next few weeks, and will have it in fedora soon thereafter.
Best Neil
On Tue, 10 Mar 2020 08:26:02 -0400 Neil Horman nhorman@redhat.com wrote:
Hey, just FYI, its still a bit nascent, but it works: https://github.com/nhorman/rng-tools
Now has an rtlsdr entropy source, which can be used to feed the kernel entropy pool.
I'll be tagging and releasing an updated rng-tools in the next few weeks, and will have it in fedora soon thereafter.
That's great news! It means lots of people will be able to take advantage of a cheap entropy source to make their systems more secure, if they want to.
kernel@lists.fedoraproject.org
-
Florian Weimer -
Laura Abbott -
Neil Horman -
stan