I am working now more on handcrafting my SD cards for arm testing. Gparted did not do a good job, allowing me to make parititions not on 'cylinder boundaries'. And the labels it created were not recognized when I mounted the drive. I had to use the disk utility to fix the labels. Anyway, to script it and to put this up on some wikis, I really need to do this by command line.
So I have looked at both fdisk and parted. Neither are for 'simple' command lines. Fdisk takes me back to my DOS days (wonder where MS got it from?).
So first I want a command that will delete all partitions on /dev/sdb
then create a partition as ext3, then one as linux-swap, and finally ext4. Of course, I understand how many MB I want each, but I am suppose to (or so from the warnings that 'fdisk -l' provided) maintain boundaries.
thanks for any pointers to the best tool(s) for this. So far my search foo has only gone to old fdisk pages.
On 08/07/2014 08:45 PM, Robert Moskowitz wrote:
I am working now more on handcrafting my SD cards for arm testing. Gparted did not do a good job, allowing me to make parititions not on 'cylinder boundaries'. And the labels it created were not recognized when I mounted the drive. I had to use the disk utility to fix the labels. Anyway, to script it and to put this up on some wikis, I really need to do this by command line.
So I have looked at both fdisk and parted. Neither are for 'simple' command lines. Fdisk takes me back to my DOS days (wonder where MS got it from?).
So first I want a command that will delete all partitions on /dev/sdb
then create a partition as ext3, then one as linux-swap, and finally ext4. Of course, I understand how many MB I want each, but I am suppose to (or so from the warnings that 'fdisk -l' provided) maintain boundaries.
thanks for any pointers to the best tool(s) for this. So far my search foo has only gone to old fdisk pages.
Hi Robert,
have a look at /sbin/cfdisk
It is part of util-linux.
Kind regards
Joachim Backes
On 08/07/2014 03:13 PM, Joachim Backes wrote:
On 08/07/2014 08:45 PM, Robert Moskowitz wrote:
I am working now more on handcrafting my SD cards for arm testing. Gparted did not do a good job, allowing me to make parititions not on 'cylinder boundaries'. And the labels it created were not recognized when I mounted the drive. I had to use the disk utility to fix the labels. Anyway, to script it and to put this up on some wikis, I really need to do this by command line.
So I have looked at both fdisk and parted. Neither are for 'simple' command lines. Fdisk takes me back to my DOS days (wonder where MS got it from?).
So first I want a command that will delete all partitions on /dev/sdb
then create a partition as ext3, then one as linux-swap, and finally ext4. Of course, I understand how many MB I want each, but I am suppose to (or so from the warnings that 'fdisk -l' provided) maintain boundaries.
thanks for any pointers to the best tool(s) for this. So far my search foo has only gone to old fdisk pages.
Hi Robert,
have a look at /sbin/cfdisk
OK. One more to study. I was also told about gdisk.
But all of these are command menu programs. Not one-liners. The nice 'one-liners' in kckstart files are just commands to anaconda, it seems. No such program and 'clearpart' or 'part'. Though I suppose I could deal with a program that read a command file of such commands.
On 07Aug2014 16:29, Robert Moskowitz rgm@htt-consult.com wrote:
On 08/07/2014 03:13 PM, Joachim Backes wrote:
have a look at /sbin/cfdisk
OK. One more to study. I was also told about gdisk.
But all of these are command menu programs. Not one-liners. The nice 'one-liners' in kckstart files are just commands to anaconda, it seems. No such program and 'clearpart' or 'part'. Though I suppose I could deal with a program that read a command file of such commands.
We have used sfdisk for a long time for cloning partition tables from one drive to another.
That cloning is done by dumping the table and then reading the dump. But that means that there is a dump format to sfdisk's output, and you can just craft the right input.
Definitely purely command line.
Cheers, Cameron Simpson cs@zip.com.au
Tomkin Coleman (tcolema@nv7.uswnvg.com) wrote: | Well, yes, get the bike but BEWARE!! | At first, you will just want to get, like, a 250 Rebel, just for commuting | and maybe a little fun, but you will wind up buying a 550, (just in case you | want to tour), and then you will stop paying any attention to the theatre | company you are working with and the graduate school you are attending, and | then you will spend all your summers touring and riding around with your | newfound biking buddies, and then you will find that you have got a real job | just so you can buy the bike of your dreams and pretty soon you will find | that your entire life has centered around your monster bike and you can only | date women who love bikes and hang with buddies who love bikes and work at a | job that supports your lifestyle and you will be incredibly happy, but always | very greasy. Shhhh! The meaning of life is supposed to be a SECRET! :-) - Bob Larson..DoD#1711..bobl@honshu.ho.att.com..1994 Suzuki RF600R
I am learning how to use parted in command line format. Unfortuately there is no such command to delete all partitions, though you kind of can do it by changing the table type, say from msdos to gpt.
Also learned that the unused 4Mb I am seeing on most SD cards is for a reason. To get on the 4Mb alignment for performance reasons. So here is some things that I have figured out:
parted /dev/sdb mklabel gpt parted /dev/sdb mkpart offset fat32 0 4 parted /dev/sdb mkpart uboot ext3 4 516
But it is still acting strange. Print is showing that ext3 partition fat32!
# parted /dev/sdb mkpart uboot ext3 4 516 Information: You may need to update /etc/fstab.
[root@lx120e ~]# parted /dev/sdb print Model: Generic- Multi-Card (scsi) Disk /dev/sdb: 7969MB Sector size (logical/physical): 512B/512B Partition Table: gpt Disk Flags:
Number Start End Size File system Name Flags 1 17.4kB 4000kB 3983kB offset 2 4194kB 516MB 512MB fat32 uboot
On 08/08/2014 01:37 AM, Cameron Simpson wrote:
On 07Aug2014 16:29, Robert Moskowitz rgm@htt-consult.com wrote:
On 08/07/2014 03:13 PM, Joachim Backes wrote:
have a look at /sbin/cfdisk
OK. One more to study. I was also told about gdisk.
But all of these are command menu programs. Not one-liners. The nice 'one-liners' in kckstart files are just commands to anaconda, it seems. No such program and 'clearpart' or 'part'. Though I suppose I could deal with a program that read a command file of such commands.
We have used sfdisk for a long time for cloning partition tables from one drive to another.
That cloning is done by dumping the table and then reading the dump. But that means that there is a dump format to sfdisk's output, and you can just craft the right input.
Definitely purely command line.
Cheers, Cameron Simpson cs@zip.com.au
Tomkin Coleman (tcolema@nv7.uswnvg.com) wrote: | Well, yes, get the bike but BEWARE!! | At first, you will just want to get, like, a 250 Rebel, just for commuting | and maybe a little fun, but you will wind up buying a 550, (just in case you | want to tour), and then you will stop paying any attention to the theatre | company you are working with and the graduate school you are attending, and | then you will spend all your summers touring and riding around with your | newfound biking buddies, and then you will find that you have got a real job | just so you can buy the bike of your dreams and pretty soon you will find | that your entire life has centered around your monster bike and you can only | date women who love bikes and hang with buddies who love bikes and work at a | job that supports your lifestyle and you will be incredibly happy, but always | very greasy. Shhhh! The meaning of life is supposed to be a SECRET! :-)
- Bob Larson..DoD#1711..bobl@honshu.ho.att.com..1994 Suzuki RF600R
On Aug 8, 2014, at 4:29 AM, Robert Moskowitz rgm@htt-consult.com wrote:
I am learning how to use parted in command line format. Unfortuately there is no such command to delete all partitions, though you kind of can do it by changing the table type, say from msdos to gpt.
Also learned that the unused 4Mb I am seeing on most SD cards is for a reason. To get on the 4Mb alignment for performance reasons.
It's 4KB, not 4MB, and it doesn't apply to SSDs, only HDD's specifically the 512e AF variety. a.) It's AF, Advanced Format, which means it has 4096 byte physical sectors, and 512e means it's emulating 512 byte sectors. So it appears as having 512 byte logical sectors, while having 4096 byte physical sectors. So one real sector has 8 logical sectors, which is where the performance problem can occur, when effectively asking the drive to update portions of a sector it causes the drive firmware to read-modify-write the sector.
parted /dev/sdb mkpart offset fat32 0 4
When I do that, parted complains. "Warning: The resulting partition is not properly aligned for best performance."
What you should do is use 1 and 5, assuming you really want only a 4MB partition. In that case the result ends up being:
Number Start End Size File system Name Flags 1 2048s 10239s 8192s offset msftdata
If you want to agree to the widely used convention, use LBA 2048 as the start. If you want to do your own thing, you could find out the erase block size for your SSD and align on those boundaries, but due to the non-transparency of the flash translation layer, there's no guarantee this means you're actually aligned on erase block boundaries: but it seems like a good idea, doesn't hurt anything, but might be a waste of effort depending on how things are actually implemented in the SSD.
parted /dev/sdb mkpart uboot ext3 4 516
But it is still acting strange. Print is showing that ext3 partition fat32!
That's probably because libparted is finding a latent FAT32 superblock. The first thing to understand about parted, is that it's crusty. It used to do a lot of stuff that it doesn't do anymore but has the legacy operands of what it used to do. It used to format and resize volumes and partitions in one go, and it no longer does this. There is no partition type code that tells parted what the filesystem is, so it looks (maybe via libblkid, not sure) for the actual partition contents to see what filesystem it is and tells you that. But it's totally superfluous information in a partitioning tool. But in parted parlance you have to know these weird things because of its legacy. Likewise it doesn't directly tell you, or let you directly tell it, what a GPT partition's partition type GUID is. You have to tell it a filesystem, and then it sets the type code accordingly, which is quite honestly maddening. But whatever you just have to accept it if you're going to use parted. Its view of the world is abstracted to the degree it'll give you a completely distorted picture, and it's why I don't like it.
So your first command with fat32, doesn't actually cause the partition to be formatted fat32. It merely set the partition type GUID to EBD0A0A2-B9E5-4433-87C0-68B6B72699C7 (Microsoft basic data). Due to poor decision making, if you're using a Fedora 20 or older version, parted uses that same type code when specifying ext3 as the file system. Out of essentially unlimited UUIDs parted devs chose to usurp a Microsoft one. It baffles me whether I'm sober or drunk. The Fedora 21 version, parted will set an ext3 partition to partition type code 0FC63DAF-8483-4772-8E79-3D69D8477DE4 (Linux filesystem) instead. And meanwhile gdisk and cgdisk have done this correctly for years.
The parted "File system" column is not a reflection of the partition type GUID, it's actually looking for filesystem superblocks to identify what the filesystem is.
Chris Murphy
On 08/08/2014 02:24 PM, Chris Murphy wrote:
On Aug 8, 2014, at 4:29 AM, Robert Moskowitz rgm@htt-consult.com wrote:
I am learning how to use parted in command line format. Unfortuately there is no such command to delete all partitions, though you kind of can do it by changing the table type, say from msdos to gpt.
Also learned that the unused 4Mb I am seeing on most SD cards is for a reason. To get on the 4Mb alignment for performance reasons.
It's 4KB, not 4MB, and it doesn't apply to SSDs, only HDD's specifically the 512e AF variety. a.) It's AF, Advanced Format, which means it has 4096 byte physical sectors, and 512e means it's emulating 512 byte sectors. So it appears as having 512 byte logical sectors, while having 4096 byte physical sectors. So one real sector has 8 logical sectors, which is where the performance problem can occur, when effectively asking the drive to update portions of a sector it causes the drive firmware to read-modify-write the sector.
It is for SDcards that I was dealing with, and my information comes from: http://www.gnu.org/software/parted/manual/parted.html
Now, we will show how to partition a low-end flash device (“low-end”, as of 2011/2012). For such devices, you should use 4MiB-aligned partition^. This command creates a tiny place-holder partition at the beginning, and then uses all remaining space to create the partition you’ll actually use:
$parted -s /dev/sdX -- mklabel msdos \ mkpart primary fat32 64s 4MiB \ mkpart primary fat32 4MiB -1s
I figured out that you really don't need the tiny place-holder, just leave it unallocated.
Good to know about HDD and SSD as that will be later....
thanks
parted /dev/sdb mkpart offset fat32 0 4
When I do that, parted complains. "Warning: The resulting partition is not properly aligned for best performance."
What you should do is use 1 and 5, assuming you really want only a 4MB partition. In that case the result ends up being:
Number Start End Size File system Name Flags 1 2048s 10239s 8192s offset msftdata
If you want to agree to the widely used convention, use LBA 2048 as the start. If you want to do your own thing, you could find out the erase block size for your SSD and align on those boundaries, but due to the non-transparency of the flash translation layer, there's no guarantee this means you're actually aligned on erase block boundaries: but it seems like a good idea, doesn't hurt anything, but might be a waste of effort depending on how things are actually implemented in the SSD.
parted /dev/sdb mkpart uboot ext3 4 516
But it is still acting strange. Print is showing that ext3 partition fat32!
That's probably because libparted is finding a latent FAT32 superblock. The first thing to understand about parted, is that it's crusty. It used to do a lot of stuff that it doesn't do anymore but has the legacy operands of what it used to do. It used to format and resize volumes and partitions in one go, and it no longer does this. There is no partition type code that tells parted what the filesystem is, so it looks (maybe via libblkid, not sure) for the actual partition contents to see what filesystem it is and tells you that. But it's totally superfluous information in a partitioning tool. But in parted parlance you have to know these weird things because of its legacy. Likewise it doesn't directly tell you, or let you directly tell it, what a GPT partition's partition type GUID is. You have to tell it a filesystem, and then it sets the type code accordingly, which is quite honestly maddening. But whatever you just have to accept it if you're going to use parted. Its view of the world is abstracted to the degree it'll give you a completely distorted picture, and it's why I don't like it.
So your first command with fat32, doesn't actually cause the partition to be formatted fat32. It merely set the partition type GUID to EBD0A0A2-B9E5-4433-87C0-68B6B72699C7 (Microsoft basic data). Due to poor decision making, if you're using a Fedora 20 or older version, parted uses that same type code when specifying ext3 as the file system. Out of essentially unlimited UUIDs parted devs chose to usurp a Microsoft one. It baffles me whether I'm sober or drunk. The Fedora 21 version, parted will set an ext3 partition to partition type code 0FC63DAF-8483-4772-8E79-3D69D8477DE4 (Linux filesystem) instead. And meanwhile gdisk and cgdisk have done this correctly for years.
The parted "File system" column is not a reflection of the partition type GUID, it's actually looking for filesystem superblocks to identify what the filesystem is.
Chris Murphy
On Aug 8, 2014, at 4:29 AM, Robert Moskowitz rgm@htt-consult.com wrote:
Unfortuately there is no such command to delete all partitions, though you kind of can do it by changing the table type, say from msdos to gpt.
I forgot to address this specifically. First, you really should delete the filesystem signature before deleting partitions. This makes the filesystem invalid, and thus things like libblkid and libparted aren't going to recognize latent (stale) filesystems. The tool for this is wipefs part of util-linux. Use it like this for example:
wipefs -a /dev/sdb[123]
That will delete the fs signatures on all file systems found on partitions 1 through 3 on disk sdb. The partition table still contains entries of course, but the filesystems in them are invalidated.
Next, if you want to get rid of all partitions, you can also use wipefs on a whole disk. This removes the signature for the partition table. Most any tool will consider it invalid, rather than broken, so it's not going to offer to fix it, it'll offer to repartition it: so in the GPT case, it gets a whole new disk identifier GUID rather than just restoring the signature. It is possible, btw to restore the signature and thus restore the partition table and all of its partitions (since those sectors aren't actually erased).
You could also blow away 34 sectors from the start and end of the drive using dd if=/dev/zero. That's a hammer.
Oh and I mentioned cgdisk (member of the gdisk family) that was wrong, it's curse-based. You want to look at sgdisk which is for use in scripts and accepts all commands from the CLI. It has a way to delete partitions individually. Note that this does not employ wipefs, so the actual filesystem contained within the partition you've deleted is still intact; and this also leaves the partition header intact, all it's doing is removing a partition table entry. sgdisk also has two zap options: one overwrites everything (sectors containing both MBR and GPT structures), the other option overwrites only the sectors containing GPT structures.
So it really depends what you want to achieve, and how arbitrary the source drives are going to be.
If you're writing a program or script you might look at python-blivet which has done a ton of work abstracting all of this stuff, if you can do what you need to do in python, then you can use sane pthyon code to do things like wipe all fs's, delete all partitions, create new GPT, add new partitions, format them. And you don't need to know the prose for the 5 different utilities to make that happen. python-blivet is actually at the core of the Fedora installer, specifically for manipulating storage (it does everything you can imagine in including create, modify, destroy LVM objects; btrfs subvolumes; bunch of md raid stuff, etc.)
http://pkgs.fedoraproject.org/cgit/python-blivet.git/
Chris Murphy
On 08/08/2014 03:00 PM, Chris Murphy wrote:
On Aug 8, 2014, at 4:29 AM, Robert Moskowitz rgm@htt-consult.com wrote:
Unfortuately there is no such command to delete all partitions, though you kind of can do it by changing the table type, say from msdos to gpt.
I forgot to address this specifically. First, you really should delete the filesystem signature before deleting partitions. This makes the filesystem invalid, and thus things like libblkid and libparted aren't going to recognize latent (stale) filesystems. The tool for this is wipefs part of util-linux. Use it like this for example:
wipefs -a /dev/sdb[123]
That will delete the fs signatures on all file systems found on partitions 1 through 3 on disk sdb. The partition table still contains entries of course, but the filesystems in them are invalidated.
Next, if you want to get rid of all partitions, you can also use wipefs on a whole disk.
wipefs -a /dev/sdb
??
thanks for all of this.
This removes the signature for the partition table. Most any tool will consider it invalid, rather than broken, so it's not going to offer to fix it, it'll offer to repartition it: so in the GPT case, it gets a whole new disk identifier GUID rather than just restoring the signature. It is possible, btw to restore the signature and thus restore the partition table and all of its partitions (since those sectors aren't actually erased).
You could also blow away 34 sectors from the start and end of the drive using dd if=/dev/zero. That's a hammer.
Oh and I mentioned cgdisk (member of the gdisk family) that was wrong, it's curse-based. You want to look at sgdisk which is for use in scripts and accepts all commands from the CLI. It has a way to delete partitions individually. Note that this does not employ wipefs, so the actual filesystem contained within the partition you've deleted is still intact; and this also leaves the partition header intact, all it's doing is removing a partition table entry. sgdisk also has two zap options: one overwrites everything (sectors containing both MBR and GPT structures), the other option overwrites only the sectors containing GPT structures.
So it really depends what you want to achieve, and how arbitrary the source drives are going to be.
If you're writing a program or script you might look at python-blivet which has done a ton of work abstracting all of this stuff, if you can do what you need to do in python,
No. I am developing my own notes on how to build up a boot card (and eventually drive) for Redsleeve on my Cubieboard2. At this stage, I tend to build then break a number of times, so having commands to build rather than having to use a GUI speeds the rebuild time. But so far no magic for fstab; I am having to put UUIDs in it.
I have been given nice scripts for F19 & F20 remixes and F21 alpha. We will see what I will need for Centos7arm development...
I will then be putting all that I did on the Redsleeve wiki.
then you can use sane pthyon code to do things like wipe all fs's, delete all partitions, create new GPT, add new partitions, format them. And you don't need to know the prose for the 5 different utilities to make that happen. python-blivet is actually at the core of the Fedora installer, specifically for manipulating storage (it does everything you can imagine in including create, modify, destroy LVM objects; btrfs subvolumes; bunch of md raid stuff, etc.)
On Aug 8, 2014, at 2:16 PM, Robert Moskowitz rgm@htt-consult.com wrote:
On 08/08/2014 03:00 PM, Chris Murphy wrote:
On Aug 8, 2014, at 4:29 AM, Robert Moskowitz rgm@htt-consult.com wrote:
Unfortuately there is no such command to delete all partitions, though you kind of can do it by changing the table type, say from msdos to gpt.
I forgot to address this specifically. First, you really should delete the filesystem signature before deleting partitions. This makes the filesystem invalid, and thus things like libblkid and libparted aren't going to recognize latent (stale) filesystems. The tool for this is wipefs part of util-linux. Use it like this for example:
wipefs -a /dev/sdb[123]
That will delete the fs signatures on all file systems found on partitions 1 through 3 on disk sdb. The partition table still contains entries of course, but the filesystems in them are invalidated.
Next, if you want to get rid of all partitions, you can also use wipefs on a whole disk.
wipefs -a /dev/sdb
That will only remove signatures from either an MBR or GPT. It will not remove signatures from filesystems. You really should remove filesystem signatures first with /dev/sdX[1234…] and then remove the partition map sig with sdX alone.
No. I am developing my own notes on how to build up a boot card (and eventually drive) for Redsleeve on my Cubieboard2. At this stage, I tend to build then break a number of times, so having commands to build rather than having to use a GUI speeds the rebuild time. But so far no magic for fstab; I am having to put UUIDs in it.
I have been given nice scripts for F19 & F20 remixes and F21 alpha. We will see what I will need for Centos7arm development...
I will then be putting all that I did on the Redsleeve wiki.
blkid will get you the UUID and filesystem type so it's a bit easier to build the fstab; otherwise to automate it with command line instead of GUI, you'd look at kickstart installs which leverages anaconda and blivet to do all of these things for you, then does the install per your requirements.
Chris Murphy
On 08/08/2014 06:11 PM, Chris Murphy wrote:
On Aug 8, 2014, at 2:16 PM, Robert Moskowitz rgm@htt-consult.com wrote:
On 08/08/2014 03:00 PM, Chris Murphy wrote:
On Aug 8, 2014, at 4:29 AM, Robert Moskowitz rgm@htt-consult.com wrote:
Unfortuately there is no such command to delete all partitions, though you kind of can do it by changing the table type, say from msdos to gpt.
I forgot to address this specifically. First, you really should delete the filesystem signature before deleting partitions. This makes the filesystem invalid, and thus things like libblkid and libparted aren't going to recognize latent (stale) filesystems. The tool for this is wipefs part of util-linux. Use it like this for example:
wipefs -a /dev/sdb[123]
That will delete the fs signatures on all file systems found on partitions 1 through 3 on disk sdb. The partition table still contains entries of course, but the filesystems in them are invalidated.
Next, if you want to get rid of all partitions, you can also use wipefs on a whole disk.
wipefs -a /dev/sdb
That will only remove signatures from either an MBR or GPT. It will not remove signatures from filesystems. You really should remove filesystem signatures first with /dev/sdX[1234…] and then remove the partition map sig with sdX alone.
I was very unclear, that is what I meant, that I got the remove sigs for each partition, but I am assuming that the command for the partition part was what I gave. So the whole set is:
wipefs -a /dev/sdb1 wipefs -a /dev/sdb2 wipefs -a /dev/sdb3 wipefs -a /dev/sdb parted /dev/sdb rm 3 parted /dev/sdb rm 2 parted /dev/sdb rm 1 parted /dev/sdb mklabel msdos (or gpt)
So much nicer what anaconda does in the kickstart with 'clearpart' ! A standalone util/script for that is handy for someone working on a USB drive of any sort.
No. I am developing my own notes on how to build up a boot card (and eventually drive) for Redsleeve on my Cubieboard2. At this stage, I tend to build then break a number of times, so having commands to build rather than having to use a GUI speeds the rebuild time. But so far no magic for fstab; I am having to put UUIDs in it.
I have been given nice scripts for F19 & F20 remixes and F21 alpha. We will see what I will need for Centos7arm development...
I will then be putting all that I did on the Redsleeve wiki.
blkid will get you the UUID and filesystem type so it's a bit easier to build the fstab; otherwise to automate it with command line instead of GUI, you'd look at kickstart installs which leverages anaconda and blivet to do all of these things for you, then does the install per your requirements.
Chris Murphy
On Aug 9, 2014, at 9:07 PM, Robert Moskowitz rgm@htt-consult.com wrote:
On 08/08/2014 06:11 PM, Chris Murphy wrote:
On Aug 8, 2014, at 2:16 PM, Robert Moskowitz rgm@htt-consult.com wrote:
On 08/08/2014 03:00 PM, Chris Murphy wrote:
On Aug 8, 2014, at 4:29 AM, Robert Moskowitz rgm@htt-consult.com wrote:
Unfortuately there is no such command to delete all partitions, though you kind of can do it by changing the table type, say from msdos to gpt.
I forgot to address this specifically. First, you really should delete the filesystem signature before deleting partitions. This makes the filesystem invalid, and thus things like libblkid and libparted aren't going to recognize latent (stale) filesystems. The tool for this is wipefs part of util-linux. Use it like this for example:
wipefs -a /dev/sdb[123]
That will delete the fs signatures on all file systems found on partitions 1 through 3 on disk sdb. The partition table still contains entries of course, but the filesystems in them are invalidated.
Next, if you want to get rid of all partitions, you can also use wipefs on a whole disk.
wipefs -a /dev/sdb
That will only remove signatures from either an MBR or GPT. It will not remove signatures from filesystems. You really should remove filesystem signatures first with /dev/sdX[1234…] and then remove the partition map sig with sdX alone.
I was very unclear, that is what I meant, that I got the remove sigs for each partition, but I am assuming that the command for the partition part was what I gave. So the whole set is:
wipefs -a /dev/sdb1 wipefs -a /dev/sdb2 wipefs -a /dev/sdb3
Sure. These can be combined as: wipefs -a /dev/sdb[123]
There are quite a few examples of this, including making LVM pvs, and Btrfs volumes, e.g. mkfs.btrfs -mraid1 -draid1 /dev/sdb[123]
And also for multiple disk partition table obliteration: wipefs -a /dev/sd[bcd]
I like wipefs because it tells you what it's removing, so you can restore it, and also has an option for backing up that which it's removing so you can easily restore it if you got sloppy with the command and bashed the wrong device. Not that this should be habit forming, like being able to make a proper gimlet.
So much nicer what anaconda does in the kickstart with 'clearpart' !
Yes and blivet makes an appearance in F21 proper although I'm not sure yet if this is an API or how it'll get leveraged. Another pony trick is system-storage-manager package, which takes the form of ssm on the command line; but this tool is post partition management, working with LVM and filesystems.
Chris Murphy
On 08/10/2014 05:20 PM, Chris Murphy wrote:
On Aug 9, 2014, at 9:07 PM, Robert Moskowitz rgm@htt-consult.com wrote:
On 08/08/2014 06:11 PM, Chris Murphy wrote:
On Aug 8, 2014, at 2:16 PM, Robert Moskowitz rgm@htt-consult.com wrote:
On 08/08/2014 03:00 PM, Chris Murphy wrote:
On Aug 8, 2014, at 4:29 AM, Robert Moskowitz rgm@htt-consult.com wrote:
Unfortuately there is no such command to delete all partitions, though you kind of can do it by changing the table type, say from msdos to gpt.
I forgot to address this specifically. First, you really should delete the filesystem signature before deleting partitions. This makes the filesystem invalid, and thus things like libblkid and libparted aren't going to recognize latent (stale) filesystems. The tool for this is wipefs part of util-linux. Use it like this for example:
wipefs -a /dev/sdb[123]
That will delete the fs signatures on all file systems found on partitions 1 through 3 on disk sdb. The partition table still contains entries of course, but the filesystems in them are invalidated.
Next, if you want to get rid of all partitions, you can also use wipefs on a whole disk.
wipefs -a /dev/sdb
That will only remove signatures from either an MBR or GPT. It will not remove signatures from filesystems. You really should remove filesystem signatures first with /dev/sdX[1234…] and then remove the partition map sig with sdX alone.
I was very unclear, that is what I meant, that I got the remove sigs for each partition, but I am assuming that the command for the partition part was what I gave. So the whole set is:
wipefs -a /dev/sdb1 wipefs -a /dev/sdb2 wipefs -a /dev/sdb3
Sure. These can be combined as: wipefs -a /dev/sdb[123]
There are quite a few examples of this, including making LVM pvs, and Btrfs volumes, e.g. mkfs.btrfs -mraid1 -draid1 /dev/sdb[123]
And also for multiple disk partition table obliteration: wipefs -a /dev/sd[bcd]
I like wipefs because it tells you what it's removing, so you can restore it, and also has an option for backing up that which it's removing so you can easily restore it if you got sloppy with the command and bashed the wrong device. Not that this should be habit forming, like being able to make a proper gimlet.
Thanks for all the help. And I figured out another way when you want a real clean card for dd of an image then xz for a compressed form for distribution. I did:
dd if=/dev/zero of=/dev/sdb bs=1M
and everything is gone. Built the partition table, the partitions, laid down the files, booted up clean, then dd the image, xz compressed it and I now have a ice compressed image I can restore to start at a clean point and can distribute to others.
Making slow, steady progress. Maybe not as good as other more experienced, but hey, for a change I will be providing something. In particular Redsleeve for Cubieboards.
So much nicer what anaconda does in the kickstart with 'clearpart' !
Yes and blivet makes an appearance in F21 proper although I'm not sure yet if this is an API or how it'll get leveraged. Another pony trick is system-storage-manager package, which takes the form of ssm on the command line; but this tool is post partition management, working with LVM and filesystems.
Chris Murphy
On Aug 10, 2014, at 5:47 PM, Robert Moskowitz rgm@htt-consult.com wrote:
Thanks for all the help. And I figured out another way when you want a real clean card for dd of an image then xz for a compressed form for distribution. I did:
dd if=/dev/zero of=/dev/sdb bs=1M
and everything is gone.
Three critiques :-) 1. This doesn't remove the backup GPT, if present, which is at the end of the drive in the last ~34 sectors. 2. It's definitely not reversible, so if you make a mistake (which you shouldn't anyway of course), maybe one of your boots flips drive designations (which does happen and is expected), you've just obliterated the wrong drive. 3. It doesn't remove filesystem signatures beyond 1MB, so they could remotely be trouble down the road.
So I still like wipefs -a for each partition, then for the whole device, better.
Built the partition table, the partitions, laid down the files, booted up clean, then dd the image, xz compressed it and I now have a ice compressed image I can restore to start at a clean point and can distribute to others.
Oooooh. Well, you will have latent sectors that previously had data on them that will be vacuumed up with this dd image because it's a sector copy. So even unused sectors that have latent data will find their data in your dd image and xz will be thwarted making it as small (and private) as possible. Who knows what data is on those old latent sectors. So now you can ignore everything else I suggested prior to this.
If SD card reacts to TRIM, which many do, you can: a.) mkfs.btrfs /dev/sdX b.) wipefs -a /dev/sdX
The default mkfs.btrfs behavior is a whole device TRIM, then it writes out a tiny amount of metadata, much less than other filesystems. Using wipefs removes that btrfs signature, and now you can build your media. On thing though is that SD card garbage collection might take a while and depends a lot on its firmware. So in the short term the dd read of the sdcard might still return latent data, until the firmware actually goes about erasing it.
Another option, is ATA security erase, but I don't know if SD cards support this. I know the above will work because I've been doing it the past few days for other testing.
Either way, the result is that there will be essentially no latent sector data, and your dd image will contain piles of zeros that xz will then compress away.
Chris Murphy
thanks. I will study this a bit.
On 08/11/2014 09:41 PM, Chris Murphy wrote:
On Aug 10, 2014, at 5:47 PM, Robert Moskowitz rgm@htt-consult.com wrote:
Thanks for all the help. And I figured out another way when you want a real clean card for dd of an image then xz for a compressed form for distribution. I did:
dd if=/dev/zero of=/dev/sdb bs=1M
and everything is gone.
Three critiques :-)
- This doesn't remove the backup GPT, if present, which is at the end of the drive in the last ~34 sectors.
- It's definitely not reversible, so if you make a mistake (which you shouldn't anyway of course), maybe one of your boots flips drive designations (which does happen and is expected), you've just obliterated the wrong drive.
- It doesn't remove filesystem signatures beyond 1MB, so they could remotely be trouble down the road.
So I still like wipefs -a for each partition, then for the whole device, better.
Built the partition table, the partitions, laid down the files, booted up clean, then dd the image, xz compressed it and I now have a ice compressed image I can restore to start at a clean point and can distribute to others.
Oooooh. Well, you will have latent sectors that previously had data on them that will be vacuumed up with this dd image because it's a sector copy. So even unused sectors that have latent data will find their data in your dd image and xz will be thwarted making it as small (and private) as possible. Who knows what data is on those old latent sectors. So now you can ignore everything else I suggested prior to this.
If SD card reacts to TRIM, which many do, you can: a.) mkfs.btrfs /dev/sdX b.) wipefs -a /dev/sdX
The default mkfs.btrfs behavior is a whole device TRIM, then it writes out a tiny amount of metadata, much less than other filesystems. Using wipefs removes that btrfs signature, and now you can build your media. On thing though is that SD card garbage collection might take a while and depends a lot on its firmware. So in the short term the dd read of the sdcard might still return latent data, until the firmware actually goes about erasing it.
Another option, is ATA security erase, but I don't know if SD cards support this. I know the above will work because I've been doing it the past few days for other testing.
Either way, the result is that there will be essentially no latent sector data, and your dd image will contain piles of zeros that xz will then compress away.
Chris Murphy
On Aug 7, 2014, at 12:45 PM, Robert Moskowitz rgm@htt-consult.com wrote:
I am working now more on handcrafting my SD cards for arm testing. Gparted did not do a good job, allowing me to make parititions not on 'cylinder boundaries'.
FWIW cylinder boundaries are legacy and irrelevant, for either SSDs (including SD cards) or HDDs. You have to go back two epochs to get to CHS being relevant. Everything is LBA these days. The boundary you want is simply 2048 for 512 byte (logical) sectors, which is a 1MB boundary. That's the easiest and works well for everything, you don't have to think about it any further. Any recent parted, fdisk, and gdisk do this.
And the labels it created were not recognized when I mounted the drive. I had to use the disk utility to fix the labels.
Not recognized? That's vague. What does happen? The volume label appears garbled, or is blank? What is "disk utility"? There's "Disks" a.k.a. gnome-disks.
Is the SD card being GPT partitioned or MBR partitioned? GPT partitions support partition names, MBR partitioning doesn't. So maybe there's a weird conflict somewhere between volume labels (a filesystem name), and partition names (a GPT only thing).
Anyway, to script it and to put this up on some wikis, I really need to do this by command line.
So I have looked at both fdisk and parted. Neither are for 'simple' command lines. Fdisk takes me back to my DOS days (wonder where MS got it from?).
So first I want a command that will delete all partitions on /dev/sdb
fdisk and gdisk have only interactive modes. parted has both command line and interactive modes, it sounds like you're only familiar with the interactive mode. But I'm not sure if it accepts MB units in command line mode; it does in interactive mode. Possibly cfdisk or cgdisk will offer what you want.
Of course, I understand how many MB I want each, but I am suppose to (or so from the warnings that 'fdisk -l' provided) maintain boundaries.
If you specify all partition start and end values in MB, or in sectors that are 2048 divisible, it's fine. You can even get away with much less, the only time alignment really matters is 4096 sector HDD's, so that's an 8 sector or 4KB alignment. It's just that 1MB or 1GB units is ultimately all we care about for partition sizes, and they happen to align.
Chris Murphy
Chris thanks for this detailed reply!
On 08/07/2014 04:14 PM, Chris Murphy wrote:
On Aug 7, 2014, at 12:45 PM, Robert Moskowitz rgm@htt-consult.com wrote:
I am working now more on handcrafting my SD cards for arm testing. Gparted did not do a good job, allowing me to make parititions not on 'cylinder boundaries'.
FWIW cylinder boundaries are legacy and irrelevant, for either SSDs (including SD cards) or HDDs. You have to go back two epochs to get to CHS being relevant. Everything is LBA these days. The boundary you want is simply 2048 for 512 byte (logical) sectors, which is a 1MB boundary. That's the easiest and works well for everything, you don't have to think about it any further. Any recent parted, fdisk, and gdisk do this.
It was fdisk that was complaining.
#fdisk -l
Disk /dev/nand: 3984 MB, 3984588800 bytes 255 heads, 63 sectors/track, 484 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xffffffff
Disk /dev/mmcblk0: 15.6 GB, 15560867840 bytes 4 heads, 16 sectors/track, 474880 cylinders Units = cylinders of 64 * 512 = 32768 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x000cd80a
Device Boot Start End Blocks Id System /dev/mmcblk0p1 33 16416 524288 83 Linux Partition 1 does not end on cylinder boundary. /dev/mmcblk0p2 16417 49184 1048576 82 Linux swap / Solaris Partition 2 does not end on cylinder boundary. /dev/mmcblk0p3 49185 465184 13312000 83 Linux Partition 3 does not end on cylinder boundary.
And the labels it created were not recognized when I mounted the drive. I had to use the disk utility to fix the labels.
Not recognized? That's vague. What does happen? The volume label appears garbled, or is blank? What is "disk utility"? There's "Disks" a.k.a. gnome-disks.
With Gparted, I removed the original MSDOS partition on the created and formated the above partitions labeling them uboot and rootfs. I removed the drive and remounted it (this wsa on my F20 notebook) and they were mounted under /run/media/rgm/ with their UUIDs, not these labels. I started up Gnome-disk-utility and sure enough, no labels, so unmounted, I labeled the partitions, and they remounted as I wanted them to be named.
Is the SD card being GPT partitioned or MBR partitioned? GPT partitions support partition names, MBR partitioning doesn't. So maybe there's a weird conflict somewhere between volume labels (a filesystem name), and partition names (a GPT only thing).
Anyway, to script it and to put this up on some wikis, I really need to do this by command line.
So I have looked at both fdisk and parted. Neither are for 'simple' command lines. Fdisk takes me back to my DOS days (wonder where MS got it from?).
So first I want a command that will delete all partitions on /dev/sdb
fdisk and gdisk have only interactive modes. parted has both command line and interactive modes, it sounds like you're only familiar with the interactive mode. But I'm not sure if it accepts MB units in command line mode; it does in interactive mode. Possibly cfdisk or cgdisk will offer what you want.
I will look more at parted.
Of course, I understand how many MB I want each, but I am suppose to (or so from the warnings that 'fdisk -l' provided) maintain boundaries.
If you specify all partition start and end values in MB, or in sectors that are 2048 divisible, it's fine. You can even get away with much less, the only time alignment really matters is 4096 sector HDD's, so that's an 8 sector or 4KB alignment. It's just that 1MB or 1GB units is ultimately all we care about for partition sizes, and they happen to align.
Further down the road, I will be setting this up on a HD or SSD, so that will be a concern before I can say I am 'done'.
On 08/07/2014 01:40 PM, Robert Moskowitz wrote:
It was fdisk that was complaining.
Those messages aren't complaints, they're just informing you. (I'm not sure any more, but I think that at one time the space between the end of the partition and the cylinder boundary couldn't be used.) I suspect that at this point, the only reason fdisk still tells you that is that none of the maintainers consider it worth the bother to remove that code. Sooner or later, somebody will get tired of seeing it and it will go away.
On Thu, 7 Aug 2014, Joe Zeff wrote:
On 08/07/2014 01:40 PM, Robert Moskowitz wrote:
It was fdisk that was complaining.
Those messages aren't complaints, they're just informing you. (I'm not sure any more, but I think that at one time the space between the end of the partition and the cylinder boundary couldn't be used.) I suspect that at this point, the only reason fdisk still tells you that is that none of the maintainers consider it worth the bother to remove that code. Sooner or later, somebody will get tired of seeing it and it will go away.
I have had a partition tool quit on me for want of cylinder alignment. Don't remember which one.
I think that sfdisk will do what OP wants.
-
Cameron Simpson -
Chris Murphy -
Joachim Backes -
Joe Zeff -
Michael Hennebry -
Robert Moskowitz