<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 07/11/2015 09:29 PM, Burn Alting
wrote:<br>
</div>
<blockquote cite="mid:1436671748.2927.4.camel@swtf.swtf.dyndns.org"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="GENERATOR" content="GtkHTML/3.32.2">
On Mon, 2015-07-06 at 08:00 -0600, Rich Megginson wrote:
<blockquote type="CITE">
<pre>On 07/03/2015 05:49 AM, Burn Alting wrote:
> Has anyone authored code to parse a 389 Directory Server's access.log
> file(s) with an aim of generating audit events based around the LDAP
> request type. Basically, take the log sequence
>
> [21/Apr/2007:11:39:51 -0700] conn=11 fd=608 slot=608 connection from
> 207.1.153.51 to 192.18.122.139
> [21/Apr/2007:11:39:51 -0700] conn=11 op=0 BIND dn="cn=Directory
> Manager" method=128 version=3
> [21/Apr/2007:11:39:51 -0700] conn=11 op=0 RESULT err=0 tag=97
> nentries=0 etime=0
> [21/Apr/2007:11:39:51 -0700] conn=11 op=1 SRCH
> base="dc=example,dc=com" scope=2 filter="(uid=bjensen)"
> [21/Apr/2007:11:39:51 -0700] conn=11 op=1 RESULT err=0 tag=101
> nentries=1 etime=1000 notes=U
> [21/Apr/2007:11:39:51 -0700] conn=11 op=2 UNBIND
> [21/Apr/2007:11:39:51 -0700] conn=11 op=2 fd=608 closed - U1
>
> And turn this into an audit event with
>
> a date/time (21/Apr/2007:11:39:51 -0700), a client location
> (207.1.153.51), server location (192.18.122.139), a user (cn=Directory
> Manager), an event (SRCH) and event metadata of (query -
> base="dc=example,dc=com" scope=2 filter="(uid=bjensen)", result set size
> - 1, timetaken = 1000 sec, etc)
>
> The logconv.pl script seems to do all sorts of analysis, but no event
> representation.
This sounds like a request for a new feature. Would you be able to
write up a description of the new feature based on
<a moz-do-not-send="true" href="http://www.port389.org/docs/389ds/design/design-template.html">http://www.port389.org/docs/389ds/design/design-template.html</a>? If so, I
will post it to the 389 wiki and assign a ticket.
</pre>
</blockquote>
Rich,<br>
<br>
Find the write up below.<br>
<br>
Regards<br>
<br>
Burn Alting<br>
</blockquote>
<br>
Thanks! <br>
<br>
<a class="moz-txt-link-freetext" href="https://fedorahosted.org/389/ticket/48222">https://fedorahosted.org/389/ticket/48222</a><br>
<a class="moz-txt-link-freetext" href="http://www.port389.org/docs/389ds/design/audit-events.html">http://www.port389.org/docs/389ds/design/audit-events.html</a><br>
<br>
<blockquote cite="mid:1436671748.2927.4.camel@swtf.swtf.dyndns.org"
type="cite">
<br>
<br>
Title<br>
-----<br>
Parse audit-able events from 389/directory server access logs<br>
<br>
Overview<br>
--------<br>
A utility is required to parse 389/directory server access logs
whose<br>
output is a well defined record (event) of the LDAP request and
any resultant<br>
responses. Each event would contain the initiating host address
and the<br>
current authenticated DN to make subsequent entity access analysis
more efficient.<br>
<br>
In essence, generate a single event for every operation (common
op=) performed<br>
for a unique connection. The events need to be well formed and
consideration given<br>
to further downstream parsing. As the access log records are well
documented,<br>
the output event should minimize changes to the content (if
changed at all).<br>
<br>
The utility would need to support time based queries. That is,
generate<br>
events between a given start and end time. Note that if the
connection<br>
and authentication occurs BEFORE the given start time, this detail<br>
still needs to decorate the event output.<br>
<br>
The utility would need to indicate if the authenticated DN or
initiating<br>
client could not be ascertained. That is, the information is NOT
in the<br>
file(s) processed.<br>
<br>
Optionally can ignore internal operations. <br>
<br>
Use Cases<br>
---------<br>
<br>
The following cases show a logfile extract and resultant parsed
output.<br>
The output is in XML. Other well formed and parsable output could
be<br>
chosen (eg json) - the intent is that downstream capability needs
to<br>
parse the information.<br>
<br>
#1<br>
Extract:<br>
<br>
[21/Apr/2009:11:39:51 -0700] conn=11 fd=608 slot=608 connection
from 207.1.153.57 to 192.18.122.139<br>
[21/Apr/2009:11:39:51 -0700] conn=11 op=0 BIND dn="cn=Directory
Manager" method=128 version=3<br>
[21/Apr/2009:11:39:51 -0700] conn=11 op=0 RESULT err=0 tag=97
nentries=0 etime=0<br>
[21/Apr/2009:11:39:51 -0700] conn=11 op=1 SRCH
base="dc=example,dc=com" scope=2 filter="(mobile=+1 123 456-7890)"<br>
[21/Apr/2009:11:39:51 -0700] conn=11 op=1 RESULT err=0 tag=101
nentries=1 etime=3 notes=U<br>
[21/Apr/2009:11:39:51 -0700] conn=11 op=2 UNBIND<br>
[21/Apr/2009:11:39:51 -0700] conn=11 op=2 fd=608 closed - U1<br>
<br>
Resultant sub-extract and Event output:<br>
<br>
[21/Apr/2009:11:39:51 -0700] conn=11 fd=608 slot=608 connection
from 207.1.153.57 to 192.18.122.139<br>
[21/Apr/2009:11:39:51 -0700] conn=11 op=0 BIND dn="cn=Directory
Manager" method=128 version=3<br>
[21/Apr/2009:11:39:51 -0700] conn=11 op=0 RESULT err=0 tag=97
nentries=0 etime=0<br>
<Event><br>
<DateTime>21/Apr/2009:11:39:51 -0700</DateTime><br>
<Client>207.1.153.57</Client><br>
<Server>192.18.122.139</Server><br>
<Connection>11</Connection><br>
<Operation>0</Operation><br>
<AuthenticatedDN>cn=Directory
Manager</AuthenticatedDN><br>
<Action>BIND</Action><br>
<Requests><br>
<Request>BIND dn=&quot;cn=Directory
Manager&quot; method=128 version=3</Request><br>
</Requests><br>
<Responses><br>
<Response>RESULT err=0 tag=97 nentries=0
etime=0</Response><br>
</Responses><br>
</Event><br>
<br>
[21/Apr/2009:11:39:51 -0700] conn=11 op=1 SRCH
base="dc=example,dc=com" scope=2 filter="(mobile=+1 123 456-7890)"<br>
[21/Apr/2009:11:39:51 -0700] conn=11 op=1 RESULT err=0 tag=101
nentries=1 etime=3 notes=U<br>
<Event><br>
<DateTime>21/Apr/2009:11:39:51 -0700</DateTime><br>
<Client>207.1.153.57</Client><br>
<Server>192.18.122.139</Server><br>
<Connection>11</Connection><br>
<Operation>1</Operation><br>
<AuthenticatedDN>cn=Directory
Manager</AuthenticatedDN><br>
<Action>SRCH</Action><br>
<Requests><br>
<Request>SRCH base=&quot;dc=example,dc=com&quot;
scope=2 filter=&quot;(mobile=+1 123
456-7890)&quot;</Request><br>
</Requests><br>
<Responses><br>
<Response>RESULT err=0 tag=101 nentries=1 etime=3
notes=U</Response><br>
</Responses><br>
</Event><br>
<br>
[21/Apr/2009:11:39:51 -0700] conn=11 op=2 UNBIND<br>
[21/Apr/2009:11:39:51 -0700] conn=11 op=2 fd=608 closed - U1<br>
<Event><br>
<DateTime>21/Apr/2009:11:39:51 -0700</DateTime><br>
<Client>207.1.153.57</Client><br>
<Server>192.18.122.139</Server><br>
<Connection>11</Connection><br>
<Operation>2</Operation><br>
<AuthenticatedDN>cn=Directory
Manager</AuthenticatedDN><br>
<Action>UNBIND</Action><br>
<Requests><br>
<Request>UNBIND</Request><br>
</Requests><br>
<Responses><br>
<Response>fd=608 closed - U1</Response><br>
</Responses><br>
</Event><br>
<br>
#2<br>
Extract:<br>
<br>
[07/May/2009:11:43:28 -0700] conn=877 fd=608 slot=608 connection
from 207.1.153.32 to 192.18.122.139<br>
[07/May/2009:11:43:28 -0700] conn=877 op=0 BIND dn="cn=Directory
Manager" method=128 version=3<br>
[07/May/2009:11:43:28 -0700] conn=877 op=0 RESULT err=0 tag=97
nentries=0 etime=0<br>
[07/May/2009:11:43:29 -0700] conn=877 op=1 SRCH base="(ou=People)"
scope=2 filter="(uid=*)"<br>
[07/May/2009:11:43:29 -0700] conn=877 op=1 SORT uid<br>
[07/May/2009:11:43:29 -0700] conn=877 op=1 VLV 0:5:0210 10:5397
(0)<br>
[07/May/2009:11:43:29 -0700] conn=877 op=1 RESULT err=0 tag=101
nentries=1 etime=0<br>
<br>
Resultant sub-extract and Event output:<br>
<br>
[07/May/2009:11:43:28 -0700] conn=877 fd=608 slot=608 connection
from 207.1.153.32 to 192.18.122.139<br>
[07/May/2009:11:43:28 -0700] conn=877 op=0 BIND dn="cn=Directory
Manager" method=128 version=3<br>
[07/May/2009:11:43:28 -0700] conn=877 op=0 RESULT err=0 tag=97
nentries=0 etime=0<br>
<Event><br>
<DateTime>07/May/2009:11:43:28 -0700</DateTime><br>
<Client>207.1.153.32</Client><br>
<Server>192.18.122.139</Server><br>
<Connection>877</Connection><br>
<Operation>0</Operation><br>
<AuthenticatedDN>cn=Directory
Manager</AuthenticatedDN><br>
<Action>BIND</Action><br>
<Requests><br>
<Request>BIND dn=&quot;cn=Directory
Manager&quot; method=128 version=3</Request><br>
</Requests><br>
<Responses><br>
<Response>RESULT err=0 tag=97 nentries=0
etime=0</Response><br>
</Responses><br>
</Event><br>
<br>
[07/May/2009:11:43:29 -0700] conn=877 op=1 SRCH base="(ou=People)"
scope=2 filter="(uid=*)"<br>
[07/May/2009:11:43:29 -0700] conn=877 op=1 SORT uid<br>
[07/May/2009:11:43:29 -0700] conn=877 op=1 VLV 0:5:0210 10:5397
(0)<br>
[07/May/2009:11:43:29 -0700] conn=877 op=1 RESULT err=0 tag=101
nentries=1 etime=0<br>
<Event><br>
<DateTime>07/May/2009:11:43:29 -0700</DateTime><br>
<Client>207.1.153.32</Client><br>
<Server>192.18.122.139</Server><br>
<Connection>877</Connection><br>
<Operation>1</Operation><br>
<AuthenticatedDN>cn=Directory
Manager</AuthenticatedDN><br>
<Action>SRCH</Action><br>
<Requests><br>
<Request>SRCH base=&quot;(ou=People)&quot;
scope=2 filter=&quot;(uid=*)&quot;</Request><br>
<Request>SORT uid</Request><br>
<Request>VLV 0:5:0210 10:5397 (0)</Request><br>
<Responses><br>
<Response>RESULT err=0 tag=101 nentries=1
etime=0</Response><br>
</Responses><br>
</Event><br>
<br>
#3<br>
Extract:<br>
[21/Apr/2009:11:39:55 -0700] conn=14 fd=700 slot=700 connection
from 207.1.153.51 to 192.18.122.139<br>
[21/Apr/2009:11:39:55 -0700] conn=14 op=0 BIND dn="" method=sasl
version=3 mech=DIGEST-MD5<br>
[21/Apr/2009:11:39:55 -0700] conn=14 op=0 RESULT err=14 tag=97
nentries=0 etime=0, SASL bind in progress<br>
[21/Apr/2009:11:39:55 -0700] conn=14 op=1 BIND
dn="uid=jdoe,dc=example,dc=com" method=sasl version=3
mech=DIGEST-MD5<br>
[21/Apr/2009:11:39:55 -0700] conn=14 op=1 RESULT err=0 tag=97
nentries=0 etime=0 dn="uid=jdoe,dc=example,dc=com"<br>
<br>
Resultant sub-extract and Event output:<br>
[21/Apr/2009:11:39:55 -0700] conn=14 fd=700 slot=700 connection
from 207.1.153.51 to 192.18.122.139<br>
[21/Apr/2009:11:39:55 -0700] conn=14 op=0 BIND dn="" method=sasl
version=3 mech=DIGEST-MD5<br>
[21/Apr/2009:11:39:55 -0700] conn=14 op=0 RESULT err=14 tag=97
nentries=0 etime=0, SASL bind in progress<br>
<Event><br>
<DateTime>21/Apr/2009:11:39:53 -0700</DateTime><br>
<Client>207.1.153.51</Client><br>
<Server>192.18.122.139</Server><br>
<Connection>14</Connection><br>
<Operation>0</Operation><br>
<AuthenticatedDN>__Anonymous__</AuthenticatedDN><br>
<Action>BIND</Action><br>
<Requests><br>
<Request>BIND dn=&quot;&quot; method=sasl
version=3 mech=DIGEST-MD5</Request><br>
</Requests><br>
<Responses><br>
<Response>RESULT err=14 tag=97 nentries=0 etime=0, SASL
bind in progress</Response><br>
</Responses><br>
</Event><br>
<br>
[21/Apr/2009:11:39:55 -0700] conn=14 op=1 BIND
dn="uid=jdoe,dc=example,dc=com" method=sasl version=3
mech=DIGEST-MD5<br>
[21/Apr/2009:11:39:55 -0700] conn=14 op=1 RESULT err=0 tag=97
nentries=0 etime=0 dn="uid=jdoe,dc=example,dc=com"<br>
<Event><br>
<DateTime>21/Apr/2009:11:39:55 -0700</DateTime><br>
<Client>207.1.153.51</Client><br>
<Server>192.18.122.139</Server><br>
<Connection>14</Connection><br>
<Operation>2</Operation><br>
<AuthenticatedDN>uid=jdoe,dc=example,dc=com</AuthenticatedDN><br>
<Action>BIND</Action><br>
<Requests><br>
<Request>BIND
dn=&quot;uid=jdoe,dc=example,dc=com&quot; method=sasl
version=3 mech=DIGEST-MD5</Request><br>
</Requests><br>
<Responses><br>
<Response>RESULT err=0 tag=97 nentries=0 etime=0
dn=&quot;uid=jdoe,dc=example,dc=com&quot;</Response><br>
</Responses><br>
</Event><br>
<br>
<br>
#4<br>
Extract:<br>
<br>
[02/Sep/2014:11:05:56 -0400] conn=35 op=1 fd=64 closed - U1<br>
[02/Sep/2014:11:05:56 -0400] conn=36 fd=64 slot=64 connection from
127.0.0.1 to 127.0.0.1<br>
[02/Sep/2014:11:05:56 -0400] conn=36 op=0 BIND dn="" method=128
version=3<br>
[02/Sep/2014:11:05:56 -0400] conn=36 op=0 RESULT err=0 tag=97
nentries=0 etime=0 dn=""<br>
[02/Sep/2014:11:05:56 -0400] conn=36 op=1 SRCH
base="dc=example,dc=com" scope=2 filter="(uid=scarter)" attrs="c"<br>
[02/Sep/2014:11:05:56 -0400] conn=36 op=1 RESULT err=0 tag=101
nentries=1 etime=0<br>
[02/Sep/2014:11:05:56 -0400] conn=36 op=2 BIND
dn="uid=scarter,ou=people,dc=example,dc=com" method=128 version=3<br>
[02/Sep/2014:11:05:56 -0400] conn=36 op=2 RESULT err=0 tag=97
nentries=0 etime=0 dn="uid=scarter,ou=people,dc=example,dc=com"<br>
[02/Sep/2014:11:05:56 -0400] conn=36 op=3 UNBIND<br>
[02/Sep/2014:11:05:56 -0400] conn=36 op=3 fd=64 closed - U1<br>
<br>
Resultant sub-extract and Event output:<br>
[02/Sep/2014:11:05:56 -0400] conn=36 fd=64 slot=64 connection from
127.0.0.1 to 127.0.0.1<br>
[02/Sep/2014:11:05:56 -0400] conn=36 op=0 BIND dn="" method=128
version=3<br>
[02/Sep/2014:11:05:56 -0400] conn=36 op=0 RESULT err=0 tag=97
nentries=0 etime=0 dn=""<br>
<Event><br>
<DateTime>02/Sep/2014:11:05:56 -0400</DateTime><br>
<Client>127.0.0.1</Client><br>
<Server>127.0.0.1</Server><br>
<Connection>36</Connection><br>
<Operation>0</Operation><br>
<AuthenticatedDN>__Anonymous__</AuthenticatedDN><br>
<Action>BIND</Action><br>
<Requests><br>
<Request>BIND dn=&quot;&quot; method=128
version=3</Request><br>
</Requests><br>
<Responses><br>
<Response>RESULT err=0 tag=97 nentries=0 etime=0
dn=&quot;&quot;</Response><br>
</Responses><br>
</Event><br>
<br>
<br>
[02/Sep/2014:11:05:56 -0400] conn=36 op=1 SRCH
base="dc=example,dc=com" scope=2 filter="(uid=scarter)" attrs="c"<br>
[02/Sep/2014:11:05:56 -0400] conn=36 op=1 RESULT err=0 tag=101
nentries=1 etime=0<br>
<Event><br>
<DateTime>02/Sep/2014:11:05:56 -0400</DateTime><br>
<Client>127.0.0.1</Client><br>
<Server>127.0.0.1</Server><br>
<Connection>36</Connection><br>
<Operation>1</Operation><br>
<AuthenticatedDN>__Anonymous__</AuthenticatedDN><br>
<Action>SRCH</Action><br>
<Requests><br>
<Request>SRCH base=&quot;dc=example,dc=com&quot;
scope=2 filter=&quot;(uid=scarter)&quot;
attrs=&quot;c&quot;</Request><br>
</Requests><br>
<Responses><br>
<Response>RESULT err=0 tag=101 nentries=1
etime=0</Response><br>
</Responses><br>
</Event><br>
<br>
[02/Sep/2014:11:05:56 -0400] conn=36 op=2 BIND
dn="uid=scarter,ou=people,dc=example,dc=com" method=128 version=3<br>
[02/Sep/2014:11:05:56 -0400] conn=36 op=2 RESULT err=0 tag=97
nentries=0 etime=0 dn="uid=scarter,ou=people,dc=example,dc=com"<br>
<Event><br>
<DateTime>02/Sep/2014:11:05:56 -0400</DateTime><br>
<Client>127.0.0.1</Client><br>
<Server>127.0.0.1</Server><br>
<Connection>36</Connection><br>
<Operation>2</Operation><br>
<AuthenticatedDN>uid=scarter,ou=people,dc=example,dc=com</AuthenticatedDN><br>
<Action>BIND</Action><br>
<Requests><br>
<Request>BIND
dn=&quot;uid=scarter,ou=people,dc=example,dc=com&quot;
method=128 version=3</Request><br>
</Requests><br>
<Responses><br>
<Response>RESULT err=0 tag=97 nentries=0 etime=0
dn=&quot;uid=scarter,ou=people,dc=example,dc=com&quot;</Response><br>
</Responses><br>
</Event><br>
<br>
[02/Sep/2014:11:05:56 -0400] conn=36 op=3 UNBIND<br>
[02/Sep/2014:11:05:56 -0400] conn=36 op=3 fd=64 closed - U1<br>
<Event><br>
<DateTime>02/Sep/2014:11:05:56 -0400</DateTime><br>
<Client>127.0.0.1</Client><br>
<Server>127.0.0.1</Server><br>
<Connection>36</Connection><br>
<Operation>3</Operation><br>
<AuthenticatedDN>uid=scarter,ou=people,dc=example,dc=com</AuthenticatedDN><br>
<Action>UNBIND</Action><br>
<Requests><br>
<Request>UNBIND</Request><br>
</Requests><br>
<Responses><br>
<Response>fd=64 closed - U1</Response><br>
</Responses><br>
</Event><br>
<br>
Design<br>
------<br>
Assuming an extension to the logconv.pl script<br>
<br>
New options:<br>
-A, --audit <ignoreinternal=yes|no><br>
Default: yes<br>
Generate well formed events of operations found in the access
log(s).<br>
Events will contain the identified connected client address and<br>
authenticated DN performing the operation. Internal
operations,<br>
if logged, will be ignored by default. Specify no to emit events<br>
for internal operations.<br>
<br>
Logic flow:<br>
for every "active" connection (ie not closed) maintain a list of
client, server and current authenticated DN.<br>
for every operation for which we have an "active" connection, emit
an event at the close of the operation.<br>
<br>
Implementation<br>
--------------<br>
Extend the logconv.pl command as it contains existing access log
file management.<br>
<br>
<br>
<br>
<br>
</blockquote>
<br>
</body>
</html>