[PATCH aggregator] Migrating rb-inotify to fssm to allow non-linux env.

This patch tries to remove Linux-specific dependency with rb-inotify and
substitute it with fssm gem which allows to run on multiple platforms.

I would really apprecite if whoever has dbomatic/dbomatic running to test it for me.
Thank you! It will make my life easier. Appreciate your time guys!

From: Ladislav Martincik <lmartinc(a)redhat.com&gt;

- This patch is simply removing 'rb-inotify' gem dependent on Linux and
substituting it with fssm which is multiplatform.
---
 src/config/environment.rb |    2 +-
 src/dbomatic/dbomatic     |   31 ++++++++++++++++---------------
 2 files changed, 17 insertions(+), 16 deletions(-)

diff --git a/src/config/environment.rb b/src/config/environment.rb
index 423f224..be6f77c 100644
--- a/src/config/environment.rb
+++ b/src/config/environment.rb
@@ -53,7 +53,7 @@ Rails::Initializer.run do |config|
   config.gem "compass-960-plugin", :lib => "ninesixty"
   config.gem "simple-navigation"
   config.gem "typhoeus"
-  config.gem "rb-inotify"
+  config.gem "fssm"
 
   config.active_record.observers = :instance_observer, :task_observer
   # Only load the plugins named here, in the order given. By default, all plugins
diff --git a/src/dbomatic/dbomatic b/src/dbomatic/dbomatic
index 8a3e968..36bea6a 100755
--- a/src/dbomatic/dbomatic
+++ b/src/dbomatic/dbomatic
@@ -22,7 +22,7 @@ $: << File.join(File.dirname(__FILE__), "../dutils")
 require 'rubygems'
 require 'dutils'
 require 'nokogiri'
-require 'rb-inotify'
+require 'fssm'
 require 'optparse'
 
 help = false
@@ -269,7 +269,7 @@ begin
   # Create one for parsing purposes.
   parser << "<events>"
 
-  notifier = INotify::Notifier.new
+  notifier = FSSM::Monitor.new
   log_file = nil
 
   if File.exists? CONDOR_EVENT_LOG_FILE
@@ -284,24 +284,25 @@ begin
       logger.info "done"
     end
 
-    # Setup inotify watch for condor event log
-    notifier.watch(CONDOR_EVENT_LOG_FILE, :modify){ |event|
-      parse_log_file log_file, parser
-    }
+    # Setup file monitor for condor event log
+    notifier.path condor_event_log_dir, File.basename(CONDOR_EVENT_LOG_FILE) do
+      update { parse_log_file log_file, parser }
+    end
 
     # if log file doesn't exist wait until it does
   else
-    notifier.watch(condor_event_log_dir, :create){ |event|
-      if event.name == "EventLog"
-        log_file = File.open(CONDOR_EVENT_LOG_FILE)
-        parse_log_file log_file, parser
-
-        # Setup inotify watch for condor event log
-        notifier.watch(CONDOR_EVENT_LOG_FILE, :modify){ |event|
+    notifier.path condor_event_log_dir do
+      create do |base, relative|
+        if relative.name == "EventLog"
+          log_file = File.open(CONDOR_EVENT_LOG_FILE)
           parse_log_file log_file, parser
-        }
+
+          notifier.path condor_event_log_dir, File.basename(CONDOR_EVENT_LOG_FILE) do
+            update { parse_log_file log_file, parser }
+          end
+        end
       end
-    }
+    end
   end
 
   while true
-- 
1.7.3.1

[Chris Lalancette]

On 11/01/10 - 01:07:43PM, lmartinc(a)redhat.com wrote:
>  From: Ladislav Martincik <lmartinc(a)redhat.com&gt;
>  
>  - This patch is simply removing 'rb-inotify' gem dependent on Linux and
>  substituting it with fssm which is multiplatform. 
I tested this, and it doesn't seem to work.  At least, I am not getting updated
events into the database even though condor is transitioning things.

In general, I'm happy to switch over to fssm for support beyond linux, but I
will NACK this patch until we have a proper Fedora RPM in review for it.
Please add fssm to the list of unpackaged dependencies at
http://deltacloud.org/page/Packaging_list

Thanks,
-- 
Chris Lalancette

[Tomas Sedovic]

On a related note, what is blocking us from doing this some other way 
than watching the log for change and parsing its contents?

Does Condor lack proper interprocess communication tools?

Thomas

On 11/01/2010 01:07 PM, lmartinc(a)redhat.com wrote:
>  This patch tries to remove Linux-specific dependency with rb-inotify and
>  substitute it with fssm gem which allows to run on multiple platforms.
>
>  I would really apprecite if whoever has dbomatic/dbomatic running to test it for me.
>  Thank you! It will make my life easier. Appreciate your time guys!
>  _______________________________________________
>  deltacloud-devel mailing list
>  deltacloud-devel(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/deltacloud-devel

[Chris Lalancette]

On 11/01/10 - 01:54:05PM, Tomas Sedovic wrote:
>  On a related note, what is blocking us from doing this some other way
>
>  than watching the log for change and parsing its contents?
>  
>  Does Condor lack proper interprocess communication tools? 
It was the fastest way to get it working.  There are some other things we can
do with this (most notably, use QMF), but watching the file for changes is
actually a pretty good way to do it, in my opinion.

-- 
Chris Lalancette

Hi Chris,

just curious, why do you think that file observation is good solution?

-- Ladislav

On Nov 1, 2010, at 2:05 PM, Chris Lalancette wrote:

>  On 11/01/10 - 01:54:05PM, Tomas Sedovic wrote:
> > On a related note, what is blocking us from doing this some other way 
> > than watching the log for change and parsing its contents?
> > 
> > Does Condor lack proper interprocess communication tools?
>  
>  It was the fastest way to get it working.  There are some other things we can
>  do with this (most notably, use QMF), but watching the file for changes is
>  actually a pretty good way to do it, in my opinion.
>  
>  -- 
>  Chris Lalancette
>  _______________________________________________
>  deltacloud-devel mailing list
>  deltacloud-devel(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/deltacloud-devel

[Chris Lalancette]

On 11/01/10 - 02:32:02PM, Ladislav Martincik wrote:
>  Hi Chris,
>  
>  just curious, why do you think that file observation is good solution? 
I guess the question is, why do you think it is not?  If you were to do
something like pipes, that would just be a file anyway, so you wouldn't really
get much efficiency gains.

Now, I agree, we eventually want to switch to QMF; that will allow us to do
it over the network, meaning we can have condor running on a different machine
from the aggregator.  But at the moment I don't really see a compelling need
to move to anything but QMF.

-- 
Chris Lalancette

[Ian Main]

On Mon, 2010-11-01 at 10:07 -0400, Chris Lalancette wrote:
>  On 11/01/10 - 02:32:02PM, Ladislav Martincik wrote:
>  > Hi Chris,
>  > 
>  > just curious, why do you think that file observation is good solution?
>  
>  I guess the question is, why do you think it is not?  If you were to do
>  something like pipes, that would just be a file anyway, so you wouldn't really
>  get much efficiency gains.
>  
>  Now, I agree, we eventually want to switch to QMF; that will allow us to do
>  it over the network, meaning we can have condor running on a different machine
>  from the aggregator.  But at the moment I don't really see a compelling need
>  to move to anything but QMF.
>   
I agree, there's nothing wrong with the file observation method we are
using now.  The format is good (XML) and we can use eg inotify or other
mechanism (kqueue on bsd) to be notified of new data.

AFAIK QMF is still not prime time for condor, though I should poke in on
that front again now as it's been some time.

	Ian

On Nov 1, 2010, at 7:52 PM, Ian Main wrote:

>  On Mon, 2010-11-01 at 10:07 -0400, Chris Lalancette wrote:
> > On 11/01/10 - 02:32:02PM, Ladislav Martincik wrote:
> >> Hi Chris,
> >> 
> >> just curious, why do you think that file observation is good solution?
> > 
> > I guess the question is, why do you think it is not?  If you were to do
> > something like pipes, that would just be a file anyway, so you wouldn't really
> > get much efficiency gains. 
Ok. Let's assume that file storage is a good solution. How do u assure that:
1) When the file gets big it will not blow your memory consumption
2) What if somebody deletes the file by accident or change its content (backup,
logrotate)
3) What if you miss some messages? Fault tolerant.

> > 
> > Now, I agree, we eventually want to switch to QMF; that will allow us to do
> > it over the network, meaning we can have condor running on a different machine
> > from the aggregator.  But at the moment I don't really see a compelling need
> > to move to anything but QMF. 
I think the discussion should about Condor being the background job processor.
IMHO choosing simple DelayedJob gem/plugin would make things much more simpler and faster
to implement from beginning.

Having sufficient message queue would make things even much better.

> > 
>  
>  I agree, there's nothing wrong with the file observation method we are
>  using now.  The format is good (XML) and we can use eg inotify or other
>  mechanism (kqueue on bsd) to be notified of new data.
>  
>  AFAIK QMF is still not prime time for condor, though I should poke in on
>  that front again now as it's been some time.
>  
>  	Ian 
My main goal for sub. of rb-inotify has been the fact that I do most of the dev. work on
my MacBook and having to comment all the time rb-inotify to be able to run server is just
painful.

-- Ladislav

[Chris Lalancette]

On 11/02/10 - 10:27:11AM, Ladislav Martincik wrote:
>  On Nov 1, 2010, at 7:52 PM, Ian Main wrote:
>  
>  > On Mon, 2010-11-01 at 10:07 -0400, Chris Lalancette wrote:
>  >> On 11/01/10 - 02:32:02PM, Ladislav Martincik wrote:
>  >>> Hi Chris,
>  >>> 
>  >>> just curious, why do you think that file observation is good solution?
>  >> 
>  >> I guess the question is, why do you think it is not?  If you were to do
>  >> something like pipes, that would just be a file anyway, so you wouldn't
> really
>  >> get much efficiency gains.
>  
>  Ok. Let's assume that file storage is a good solution. How do u assure that:
>  1) When the file gets big it will not blow your memory consumption 
Could be a problem, in theory.  Disk storage is so unbelievably cheap, though,
that I'm not inclined to worry too much about it.  When we switch to QMF, this
will go away.

>  2) What if somebody deletes the file by accident or change its
> content (backup, logrotate) 
We could handle this (though we don't today).  Inotify can also tell you when
your file goes away, so we could try to handle that intelligently.  That being
said, you have similar problems with almost anything; what happens if your
network socket closes unexpectedly?

>  3) What if you miss some messages? Fault tolerant. 
We already handle this.  We store the place in the file that we last parsed
from, and on startup we look for messages we missed.  Again, QMF will also
handle this.

>  
>  >> 
>  >> Now, I agree, we eventually want to switch to QMF; that will allow us to do
>  >> it over the network, meaning we can have condor running on a different machine
>  >> from the aggregator.  But at the moment I don't really see a compelling
> need
>  >> to move to anything but QMF.
>  
>  I think the discussion should about Condor being the background job processor.
>  IMHO choosing simple DelayedJob gem/plugin would make things much more simpler and faster
> to implement from beginning.
>  
>  Having sufficient message queue would make things even much better. 
I know you weren't here when we made this decision, but suffice it to say that
removing condor and going with something else is absolutely not an option.
Condor is known to scale well (to hundreds of thousands of jobs), and that
is what we want to target for our scheduling.

>  
>  >> 
>  > 
>  > I agree, there's nothing wrong with the file observation method we are
>  > using now.  The format is good (XML) and we can use eg inotify or other
>  > mechanism (kqueue on bsd) to be notified of new data.
>  > 
>  > AFAIK QMF is still not prime time for condor, though I should poke in on
>  > that front again now as it's been some time.
>  > 
>  > 	Ian
>  
>  My main goal for sub. of rb-inotify has been the fact that I do most of the dev. work on
>  my MacBook and having to comment all the time rb-inotify to be able to run server is just
> painful. 
Right.  I'm fine with replacing rb-inotify with fssm, I just want to make sure
we have the packages and it works before we do that.

-- 
Chris Lalancette