Hi Ian

I must apologize as I forgot to mention that I do in fact use browse mode. For the windows->Linux migration, it helps because we are moving to a case sensitive world, and people need to know the exact case of the mount points.

I will still try out submounts as you suggested to see if the mount storm is any better than the hierarchial mounts.

Thanks once again.


On Sat, Dec 7, 2019, 11:44 PM Ian Kent <ikent@redhat.com> wrote:
On Fri, 2019-12-06 at 07:34 -0500, Oguzhan Eris wrote:
> Thank you for thinking about this.  I will investigate the submounts
> you suggested as well as the NFS junctions.
>
> I am on the latest centos/rhel 7.x line

So you should have "browse_mode = no" in your installed configuration
unless you're nuked the default installed config and created a new one
from scratch without that option in a provisioning system or you retain
and old /etc/sysconfig/autofs that overrides it.

So, supposing this isn't the case, unless you add a "browse" option
to mounts it should be disabled so that mount point directories aren't
pre-created for indirect mount map keys.

In that case find(1) won't see the mount point directories and won't
be able to try and access them.

>
> On Fri, Dec 6, 2019, 2:39 AM Ian Kent <ikent@redhat.com> wrote:
> > On Thu, 2019-12-05 at 23:09 -0500, Oguzhan Eris wrote:
> > > Thanks for the reply.
> > > My file based maps are as such:
> > > basically blank auto.master  that includes auto.master.d
> > > /etc/auto.master.d/foo.autofs:
> > > /foo/x   /etc/automap/x.txt
> > > /foo/x/dev   /etc/automap/x_dev.txt
> > > foo/x/prod  /etc/automap/x_prod.txt
> > > /foo/z   /etc/automap/z.txt
> > >
> > > /etc/automap/x.txt:
> > > dir1 -fstype=nfs4 server1:/share_dir1
> > > dir2 -fstype=nfs4 server2:/share_dir2
> > >
> > > /etc/automap/x_dev.txt:
> > > dir3  -fstype=nfs4 devserver1:/share_dir3
> > > dir4  -fstype=nfs4 devserver2:/share_dir4
> > >
> > > /etc/automap/x_prod.txt:
> > > dir3  -fstype=nfs4 prodserver1:/share_dir3
> > > dir4  -fstype=nfs4 prodserver2:/share_dir4
> > >
> > > /etc/automap/z.txt:
> > > dir5 -fstype=nfs4 server5:/share_dir5

Following on from above this should be able to be done by using
autofs submounts so that offsets aren't used at all and together
with not enabling mount point browsing this should go a long way
to resolving the mount storm problem you see.

Aside: stat family system calls aren't meant to trigger automounts
(really only applies to indirect mounts, direct mounts and offset
mounts always trigger mounts because of the way they behave) but
way back when this was done the statfs(2) system call was missed
and continued to trigger mounts. And IIRC find(1) uses statfs()
during tree traversal and we get mount storms. Now it's too late
to change that but there's also a case that statfs() should trigger
mounts (rather than requiring a trailing "/" as with making other
stat family calls trigger mounts) since a statfs() of an autofs fs
path is pretty much useless ...

Now I haven't tested this at all but from the above this is what
I think might help.

In the master map (auto.master or your master map fragment) use:
/foo    /etc/automap/auto.foo

where auto.foo is:
x  --fstype=autofs /etc/automap/auto.x
z  --fstype=autofs /etc/automap/auto.z

and auto.x is:
dir1 -fstype=nfs4 server1:/share_dir1
dir2 -fstype=nfs4 server2:/share_dir2
dev    --fstype=autofs  /etc/automap/auto.x_dev
prod   --fstype=autofs  /etc/automap/auto.x_prod

and auto.x_dev is:
dir3  -fstype=nfs4 devserver1:/share_dir3
dir4  -fstype=nfs4 devserver2:/share_dir4

and auto.x_prod is:
dir3  -fstype=nfs4 prodserver1:/share_dir3
dir4  -fstype=nfs4 prodserver2:/share_dir4

and finally auto.z is:
dir5 -fstype=nfs4 server5:/share_dir5

It's not as compact and simple as using offsets allone but it
should prevent the mount storm.

Converting this to be stored in LDAP should be straight forward
too since you've already had to do that for some of the maps.

Ian

> >
> > Ok, thanks for that.
> >
> > >
> > >
> > > Such a map might not make a lot of sense if you were designing
> > from
> > > scratch, but what I'm actually doing is mapping our Windows DFS
> > > namespace "\\foo"  in Active Directory where a lot of apps have
> > been
> > > written to use just one global namespace, to now be available
> > > under linux in the same way at /foo .  This will greatly minimize
> > the
> > > effort when having these apps migrate and run under linux, or in
> > some
> > > situations, be completely cross platform apps with very minor
> > > settings to them to make them work under windows vs. linux
> > (mostly
> > > python apps)
> >
> > LOL, the why people do what they do is not really my place to
> > question.
> > The fact you feel you need to do it is enough for me.
> > But, for various reasons, it may need to be done a bit differently.
> >
> > >
> > > Anyway,  so as I have it above, as long as my indirect map in
> > > foo.autofs  is ordered the way I have above,  then everything
> > works
> > > fine, and I will end up with:
> > >
> > > /foo/x
> > > /foo/x/dir1
> > > /foo/x/dir2
> > > /foo/x/dev/
> > > /foo/x/dev/dir3
> > > /foo/x/dev/dir4
> > > /foo/x/prod/
> > > /foo/x/prod/dir3
> > > /foo/x/prod/dir4
> > > /foo/z
> > >
> > > When I model this in LDAP (via sssd) however, I can see the maps
> > > perfectly fine in automap -m   
> > > but the order in which they're returned results in some of the
> > parent
> > > paths masking their nested children.
> > > i.e.
> > > if /foo/x/dev   gets mounted first,  and then /foo/x/prod,
> > followed
> > > by /foo/x
> > > It will result in  me seeing just
> > > /foo/x/dir1
> > > /foo/x/dir2
> > >
> > > The moment I umount /foo/x   I will then see
> > > /foo/x/dev/...  and /foo/x/prod/...
> > >
> > > Hopefully that makes sense.
> >
> > Yes, I got the idea of the problem you were having at the start.
> >
> > >
> > > I had previously done hierarchical maps with something like:
> > >
> > > auto.master
> > > /foo /etc/automount/foo.txt
> > >
> > > /etc/automount/foo.txt:
> > > x    \
> > >   dir1 ... \
> > >   dir2  .... \
> > >   dev/dir3 ....\
> > >   dev/dir4 ... \
> > >   prod/dir3 ...\
> > >   prod/dir4 ...
> > >
> > > And that worked, but it had the huge burden that any traversal to
> > a
> > > path under /foo/x  meant that ALL mounts got mounted at once,
> > which
> > > was quite undesirable.
> >
> > Yes, that can happen with this type of map entry.
> >
> > This sort of construct probably could be done using submounts
> > similar to the example I gave previously.
> >
> > I guess I should ask what version of autofs your using and on what
> > distribution?
> >
> > Do you set browse_mode in your autofs configuration?
> >
> > If browse_mode is set to "no" (if it's not present in the config
> > it's default is yes) then mount point directories in indirect
> > mounts won't be seen until mounted so they won't get mounted
> > when the tree is scanned ... but with that construct you have
> > above those offsets will behave like direct mounts which will
> > be present and will respond to scans.
> >
> > It may be possible to break this tree down to use submounts which
> > could prevent (at least some of the sensitivity) to the mount on
> > scan problem you have.
> >
> > I'll need to think about it.
> >
> > >
> > > Would you  have a recommendation on how this might be better
> > authored
> > > in ldap?   As I said, I must unfortunately follow the namespace
> > as
> > > is.
> >
> > LDAP, and NIS and NISPLUS all have no ordering guarantee.
> >
> > Strictly speaking I don't allow nesting in anything other than
> > those
> > hierarchical mount map entries but I don't explicitly check for it.
> >
> > And I don't really want to put that sort of a limitation into
> > autofs.
> >
> > But the problem happens all to often (with hierarchical map
> > entries)
> > that someone rips out an export in the midst of a tree that's in
> > use
> > and the mount ends up not behaving properly. Then, after the kernel
> > mounts list gets disjointed they demand to know why this happened
> > and I can't give an answer because something like "you did
> > something
> > you should have known not to do" doesn't go down all that well.
> >
> > Inevitably I can't reproduce these types of problem so I can't even
> > work out if I can improve things ...
> >
> > Also, while I'm tempted to just sort the master map list I'm pretty
> > sure that could break other functionality people might have come to
> > expect to behave a certain way.
> >
> > Consequently I would like to find another way to achieve this if we
> > can. I'll give that some thought.
> >
> > Your mounting NFS mounts, could you use the NFS cross-mount
> > automounting here?
> >
> > I'm pretty sure nfs-utils (but perhaps fairly recent releases) have
> > a utility to set basic mount junctions. Have you investigated this,
> > autofs should play well with the NFS automounting?
> >
> > I guess that takes away the benefit of being able to manage your
> > mounts in one place using one method ...
> >
> > I'll have a bit of a think about this one.
> > Ian
> >
> > >
> > > Thanks again
> > > 
> > >
> > >
> > >
> > > On Wed, Dec 4, 2019 at 7:20 PM Ian Kent <ikent@redhat.com> wrote:
> > > > On Wed, 2019-12-04 at 11:36 +0100, Pavel Březina wrote:
> > > > > On 11/30/19 5:41 PM, Oguzhan Eris wrote:
> > > > > > Hi everyone.
> > > > > >
> > > > > > First off, thanks to everyone who's ever worked on SSSD.
> > It's
> > > > > > easily in my top 5 favorite FOSS projects out there.
> > > > > >
> > > > > > I am not sure if this is the right way to ask for an
> > > > enhancement,
> > > > > > or whether I should file an issue on GitHub,  but I am
> > running
> > > > into
> > > > > > an issue that's described in this Red Hat page   
> > > > > > https://access.redhat.com/solutions/3673501 (login
> > required)
> > > > > >
> > > > > > Basically for an automount map where I need nested top
> > level
> > > > paths:
> > > > > >
> > > > > > /mnt/foo
> > > > > > /mnt/foo/bar
> > > >
> > > > Those look like master map entries or perhaps direct mount map
> > > > entries.
> > > >
> > > > Which is it?
> > > >
> > > > In either case nesting mounts is not supported so sorting is
> > not
> > > > needed.
> > > >
> > > > Nesting of mounts can only be done by using mount map entries
> > > > with offsets.
> > > >
> > > > > >
> > > > > > each defined by their own map objects.  SSSD does not
> > handle
> > > > this
> > > > > > (neither does LDAP directly from autofs) because the return
> > map
> > > > > > from LDAP is unsorted, and if the maps are provided to
> > autofs
> > > > > > ordered as:
> > > >
> > > > So each has their own map, but you haven't provided a complete
> > > > master map entry or map examples so I still can't tell what you
> > > > actually want to do.
> > > >
> > > > I'll assume these are master map entries ... and the maps are
> > > > called auto.foo and auto.bar, and they are indirect mount maps,
> > > > and I'll use file maps for the example.
> > > >
> > > > In the master map:
> > > > /mnt    /etc/auto.foo
> > > >
> > > > In /etc/auto.foo:
> > > >
> > > > foo     /       foo.server.net:/path/to/foo
> > > >         /bar -type=autofs /etc/auto.bar
> > > >
> > > > might be something like what you need to do.
> > > >
> > > > Allowing nesting in offset mounts this problematic enough, due
> > > > to the fact that mount dependency changes can (and do) cause
> > > > problems that are extremely hard to resolve, nesting won't be
> > > > supported in any other way.
> > > >
> > > > > >
> > > > > > /mnt/foo/bar
> > > > > > /mnt/foo
> > > > > >
> > > > > > the /mnt/foo map masks the previous /mnt/foo/bar map  and
> > > > you'll
> > > > > > only get the entries from /mnt/foo
> > > > > >
> > > > > > Using file based mount maps, one can easily sort the top
> > level
> > > > > > maps, and get around this issue.
> > > > > > Would it be possible to have SSSD return the maps from LDAP
> > > > query
> > > > > > in a sorted way?   There is an LDAP control that most LDAP
> > > > servers
> > > > > > support to return a sorted output, (
> > > > > > https://ldapwiki.com/wiki/Server%20Side%20Sort%20Control )
> > but
> > > > with
> > > > > > so many clients and a large list, this might be better left
> > to
> > > > the
> > > > > > client to do instead.
> > > > > >
> > > > > > I'm happy to help out if someone can point me in the right
> > > > > > direction in the code.
> > > > >
> > > > > SSSD is just a data provider, if some sorting is needed I do
> > not
> > > > > think
> > > > > it should be done in SSSD (unless autofs interface says so)
> > but
> > > > > rather
> > > > > in autofs itself.
> > > > >
> > > > > CCing Ian, do you have any thoughts on this?
> > > >
> > > > Thanks Pavel, some more specific information about the use case
> > > > might still be needed, we'll see.
> > > >
> > > > Ian
> > > >
> >