2010-07-26
Bootstrapping "opengoupware.us"
2010-07-08
Coils: Features merged into "default"
- The scheduler service no longer depends on the ticktock heartbeat. It checks the run queue in its work method.
- Format(s) now support logging of rejected records to a buffer. This feature is available via a new parameter on the readAction: "rejectionsLabel". This is documented in the readAction's wiki page.
- The above motivated me to abstract how ActionCommand goes about creating [output] messages. It is now simple for an action to create additional messages [beyond it's 'default' output message] by just calling the store_in_message(label, wfile, mimetype='application/octet-stream') method.
- New CLI tools: "coils-list-schedule" and "coils-unschedule-process" that let the administrator view and modify the workflow scheduler from the command line. Yes, we still need a "coils-schedule-process".
- getopt isn't the greatest command line option parser, would be great if someone would look into swapping that out with something better.
- The above motivated me to add an initialize_tool(name, argv, arguments=['',[]]) function to coils.core.utility. This provides a simple way for a tool [vs. a service] to bootstrap itself onto the OpenGroupware Coils service bus so that it can receive callbacks, etc... Return value is an AdministrativeContext (with a registered Broker for IPC) and dictionary of unconsumed command line parameters. initialize_tool automatically consumes the --store=, --add-bundle=, and --ban-bundle parameters in order to initialize the Coils environment.
2010-05-24
Keeping an OIE Route After Completion
from coils.core import *
ctx = AdministrativeContext()
r = ctx.run_command('route::get', name='MTAStockOrder_TEST')
ctx.property_manager.set_property(r, 'http://www.opengroupware.us/oie', 'preserveAfterCompletion', 'YES')
ctx.commit() The above will set the value of {http://www.opengroupware.us/oie}preserveAfterCompletion to "YES" for the route named "MTAStockOrder_TEST". The property manager will update the value of the property if such a property already exists, or it will create a new property with the specified value.
2010-04-25
Performing PROPFIND from Python
- import httplib, urllib2, base64, sys
- from lxml import etree
- PROPFIND = u'''<?xml version="1.0" encoding="utf-8"?>
- <propfind xmlns="DAV:">
- <prop>
- <getetag/>
- </prop>
- </propfind>'''
- SERVER_HOST = '127.0.0.1'
- SERVER_PORT = 8080
- SERVER_PATH = '/dav/Contacts'
- CLIENT_AGENT = 'Whitemice rules/so very true'
- auth_string = 'Basic {0}'.format(base64.encodestring('adam:fred123')[:-1])
- urllib2.install_opener(urllib2.build_opener(urllib2.HTTPHandler()))
- connection = httplib.HTTPConnection(SERVER_HOST, SERVER_PORT)
- connection.putrequest('PROPFIND', SERVER_PATH)
- connection.putheader('Authorization', auth_string)
- connection.putheader('User-Agent', CLIENT_AGENT)
- connection.putheader('Depth', 1)
- connection.putheader('Content-Length', str(len(PROPFIND)))
- connection.endheaders()
- connection.send(PROPFIND)
- response = connection.getresponse()
- if response.status == 207:
- data = response.read()
- connection.close()
- response = None
- namespace_prefix_map = { 'D' : 'DAV:' }
- document = etree.fromstring(data)
- for path in document.xpath('/D:multistatus/D:response/D:href/text()',
- namespaces=namespace_prefix_map):
- if (path != SERVER_PATH):
- connection = httplib.HTTPConnection('127.0.0.1', 8080)
- connection.putrequest('GET', path)
- connection.putheader('Authorization', auth_string)
- connection.putheader('User-Agent', CLIENT_AGENT)
- connection.endheaders()
- response = connection.getresponse()
- if response.status != 200:
- print 'Error retrieving {0}'.format(path)
- sys.exit(1)
This will make the PROPFIND for all the items in the collection, requesting the getetag property, and then from the response perform an HTTP GET for every item. Even better would be if it requested isCollection and knew better than to perform GETs on collections. I hope this simple example will be of use to someone.
2010-03-17
Disabling GNOME Automount
gconftool-2 --type bool --set /apps/nautilus/preferences/media_automount false
gconftool-2 --type bool --set /apps/nautilus/preferences/media_automount true
2010-02-10
Configuring OpenLDAP's dynlist in cn=config
dn: cn=modules{0}, cn=config
olcModuleLoad: {0}accesslog.la
olcModuleLoad: {1}auditlog.la
olcModuleLoad: {2}constraint.la
olcModuleLoad: {3}dynlist.la
olcModuleLoad: {4}memberof.la
olcModuleLoad: {5}ppolicy.la
olcModuleLoad: {6}refint.la
olcModuleLoad: {7}seqmod.la
olcModuleLoad: {8}syncprov.la
olcModuleLoad: {9}sssvlv.la
olcModuleLoad: {10}translucent.la
olcModuleLoad: {11}unique.la
olcModuleLoad: {12}back_monitor.la
olcModulePath: /usr/lib/openldap2.4
objectClass: olcModuleList
cn: modules{0}
NOTE: Double check that the "olcModulePath" is the absolute path to the directory containing the OpenLDAP modules.
Step#2: Check that your schema has the groupOfURLs objectclass defined. If you attempt to configure the dynlist module without that schema available you still crash slapd. To define the dynamicGroup schema (if it is missing) you can import the following LDIF into your cn=config:
dn: cn=dynamicGroup, cn=schema, cn=config
olcObjectClasses: {0}( 2.16.840.1.113730.2.33 NAME 'groupOfURLs'
SUP top STRUCTURAL MUST cn MAY ( memberURL $ businessCat
egory $ description $ o $ ou $ owner $ seeAlso ) )
olcAttributeTypes: {0}( 2.16.840.1.113730.1.198 NAME 'memberURL'
DESC 'Identifies an URL associated with each member of a group. Any type
of labeled URL can be used.' SUP labeledURI )
objectClass: olcSchemaConfig
cn: dynamicGroup
Step#3: Create an olcOverlayConfig object in the scope of your Dit database. For example, our Dit is the 1st HDB database on the server so the appropriate object is:
dn: olcOverlay=dynlist,olcDatabase={1}hdb,cn=config
objectClass: olcOverlayConfig
objectClass: olcDynamicList
olcOverlay: dynlist
Step#4: Now you are ready to actually use the dynlist module. A common use-case is to create dynamic mail alias objects; with dynlist you don't need to maintain mail aliases, they will automatically contain everyone who matches the relevent criteria. Provided you use the traditional nisMailAlias objectclass in order to define mail aliases adding the attribute -
olcDlAttrSet: {0}nisMailAlias labeledURI rfc822mailmember:mail- to "olcOverlay=dynlist,olcDatabase={1}hdb,cn=config" will enable dynamic mail aliases. Specifically any nisMailAlias containing a labeledURI attribute will be expanded by the query specified in that attribute. The results of that query will be rewritten by the optional rfc822mailmember:mail clause which will rename the mail attributes resulting from the query into the rfc822mailmember attribute required by consumers of the nisMailAlias objects. So rather than populating the mail alias object with rfc822mailmember attributes manually, you extend the object with the auxilliary labeledURIObject objectclass and define the query in the labeledURI attribute.
dn: cn=gr_parts, ou=ListAliases, ou=Aliases, ou=Mail, ou=SubSystems, o=Morrison Industries,c=US
mail: gr_parts@morrison-ind.com
labeledURI: ldap:///ou=People,ou=Entities,ou=SAM,o=Morrison Industries,c=US?mail?one?(&(morrisonactiveuser=Y)(objectclass=morrisonuser)(departmentNumber=*P*)(morrisonbranch=GRD))
objectClass: nisMailAlias
objectClass: top
objectClass: labeledURIObject
cn: gr_parts
The object will immediately populate with rfc822mailmember attributes derived from the mail attribute of those objects matching the specified filter: "(&(morrisonactiveuser=Y)(objectclass=morrisonuser)(departmentNumber=*P*)(morrisonbranch=GRD))". The third parameter, "one", of the URI is the scope of the query so only objects immediately subordinate to "ou=People,ou=Entities,ou=SAM,o=Morrison Industries,c=US" are candidates for the filter.
2010-01-21
Python Curses In Action, even on AIX.

So, what if you have an old COBOL application that you want to integrate with some web services or a website? You'd need a "green-screen" application that can take that COBOL applications temporary output file, send it up to the web service, wait for a response, replace the temporary file, and then return control to the COBOL application (or whatever "green screen" application comes next). It sounds simpler than it is in practice: what if the web service takes awhile to complete [or has human involvement on the other end?!] or the web service isn't available right at the moment you try to send your request? For a real production environment your "green screen" application would need to deal with all of those things.
And you need that "green screen" application to work on both LINUX and AIX!
Fortunately pware provides Python 2.6 for AIX, including curses! The curses library and respective Python module provide a surprisingly easy way to create professional looking TUI (Text User Interface) applications like the solution required in our example. This particular application takes them temporary file and submits it via a WebDAV PUT operation to the workflow engine provided by OpenGroupware Coils. The route in the workflow engine reformats the dreadful output of the COBOL application into the required format (using the format support in route Read and Write operations). The client detects the route is complete by watching the URL specified in the header of the initial PUT, and then downloads the required data. If anything goes wrong along the way the client can retry, or provide the user the option to abort. All along the way the client provides detailed feedback to the user and a familiar dialog-box and prompt interface when feedback is required.
2009-09-11
New OGo Help Desk Feature
ALTER TABLE job ADD COLUMN owner_id INT;
UPDATE job SET owner_id = creator_id;
The database scripts pg-build-schema.psql and pg-update-1.x-to-5.5.psql have been updated. NOTE: Be careful not to run this part of pg-update-1.x-to-5.5.psql more than once or you risk modifying actual data in your database if you use the OGoHelpDeskRoleName default.
This feature allows a member of the team whose named is defined in the OGoHelpDeskRoleName default to create tasks with an owner other than themselves. Delegated and archived task lists now display tasks based on owner rather than creator. By default the owner is the creator so this has no effect on normal task behaviour. Currently the help desk feature, setting of the owner to another user, is only available via the zOGI API. The modifications to the zOGI API are documented on the Task entity.
If your help desk team name contains spaces be sure to use proper quoting - Defaults write NSGlobalDomain OGoHelpDeskRoleName '"all intranet"' - or you may not get the results you expect. Obviously if you set OGoHelpDeskRoleName to "all intranet" all users will be able to create tasks on behalf of other users.
2009-07-26
Service BASIC HTTP Authentication with Python
import BaseHTTPServer
class HTTPServer(BaseHTTPServer.HTTPServer):
pass
from coils.net.handler import HTTPRequestHandler
from coils.net.server import HTTPServer
....
HTTP_HOST = 'localhost'
HTTP_PORT = 8080
httpd = HTTPServer((HTTP_HOST, HTTP_PORT), HTTPRequestHandler)
httpd.serve_forever()
Your app will almost certainly provide a custom HTTPRequestHandler to delegate the requests to whatever logic your application provides. In the BaseHTTPRequestHandler a GET request is handed to do_GET, a POST request to do_POST, etc... In our case the object that handles the request will decide what to do based on the request type so we channel all requests to our generic process_request method. process_request looks up the object targeted by the request via marshall_handler() [not shown] and then calls that objects do_request method. The important part for authentication is to catch the exception raised by the object handling the request if that object thinks the request is not authenticated and requires authentication. In this case that is the COILS' AuthenticationException; your application has to provide something equivalent. In order to make the client try again with authentication you need to send a "WWW-Authenticate" header telling the client to use basic authentication and what realm to use. See RFC2617 for details on Basic authentication in general
class HTTPRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
def process_request(self):
"""Respond to a request"""
try:
""" find the object mapped to the specified request
The marshall_handler() is a COILS specific things so it isn't
shown in this example. """
handler = self.marshall_handler()
handler.do_request(self)
except AuthenticationException, err:
""" An AuthenticationException has an error code of 401
We need to add an authentication header so the client will know
to respond the the failure with the appropriate credentials
TODO: Provide a digest realm once digest authentication is supported
"""
self.send_response(err.error_code())
self.send_header('WWW-Authenticate', 'Basic realm="OpenGroupware COILS"')
self.end_headers()
self.wfile.write('Authentication failure')
except CoilsException, err:
# An Coils Exception has an error code of 500
self.send_response(err.error_code(), err.error_text())
self.end_headers()
except Exception, err:
# Yikes, something generic web very wrong
self.send_response(500, err)
self.end_headers()
def do_GET(self):
"""Respond to a GET request."""
self.process_request()
def do_POST(self):
"""Respond to a POST request"""
self.process_request()
In order to actually process the authentication request COILS uses an Authenticator object. Our DBAuthenticator object provides basic authentication against accounts with passwords stored directly in the database [verses accounts from LDAP or trusting an external authorization mechanism such as Kerberos]. Explanation for the steps to process a Basic authentication operation have been added to the code example:
from base64 import b64decode
from crypt import crypt
class DBAuthenticator(Authenticator):
def _authenticate(self, context, request):
Authenticator._authenticate(self, context, request)
authorization = request.headers.get('authorization')
if (authorization == None):
raise AuthenticationException('Authentication Required')
(kind, data) = authorization.split(' ')
if (kind == 'Basic'):
# Authentication method is "Basic"
(username, _, password) = b64decode(data).partition(':')
""" This method provided by the parent class goes to the ORM and
retrieves the account object for the specified username. It
will throw an authentication exception if no such username is
found (user entered it wrong?) or a generic Coils Exception if
multiple objects match the username (that doesn't make sense) -
either will stop the authentication process, but the authentication
exception should reprompt the client to try again. """
account = Authenticator._getLogin(self, username)
secret = account.password
if (secret == crypt(password, secret[:2])):
# Password matches, user is authenticated
self.loginId = account.objectId
self.login = account.login
else:
# Password does not match, authentication failes
raise AuthenticationException('Incorrect username or password')
else:
# Authorization header indicated an authentication type other than BASIC
CoilsException('Unsupported HTTP Authenticated Mech')
See, that easy easy.
2009-07-08
Database Changes to OpenGroupware v5.5
For those building from trunk, or currently pulling packages from OBS, there is a new table required as of r2256.
CREATE TABLE ctags (
entity VARCHAR NOT NULL,
ctag INTEGER NOT NULL DEFAULT 0
);
INSERT INTO ctags (entity) VALUES ('Person');
INSERT INTO ctags (entity) VALUES ('Enterprise');
INSERT INTO ctags (entity) VALUES ('Date');
INSERT INTO ctags (entity) VALUES ('Job');
INSERT INTO ctags (entity) VALUES ('Team');
This is part of adding ctag support to OGo (specifically ZideStore). ctags allow a client to *very* quickly detect if the contents of a collection have changed; such as the /public/Contacts folder. Thus avoiding doing a PROPFIND on the entire folder to determine if a re-sync is needing. ctag support is not complete or working yet, but without this table a server post-r2256 will fails some operations with a database error.
The subversion repository contains an update script for migrating a v5.4 database to v5.5.
2009-06-29
Python XML-RPC Presentation
Presentation File
While it is admittedly a runt of an RPC solution I've found XML-RPC to be very useful and have used it extensively in both my work on OpenGroupware and other projects.
2009-06-16
Logging *USEFUL* Web Access To PostgreSQL
cache_access_log syslog:LOG_LOCAL4
After a restart SQUID messages show up in the LOCAL4 facility (table: facility_local4) with the priority of "info". Then I created a table to hold the parsed messages:
CREATE TABLE proxy_audit (
timestamp TIMESTAMP,
elapsed INT,
source VARCHAR(40),
status VARCHAR(15),
size INT,
method VARCHAR(15),
url VARCHAR(128),
domain VARCHAR(60),
identity VARCHAR(25),
mimetype VARCHAR(45));
Last a trigger on the LOCAL4 log parses incoming messages into this table:
CREATE OR REPLACE FUNCTION p_proxy_audit() RETURNS trigger AS '
BEGIN
IF (new.hostname = ''hod-a'' AND new.priority = ''info'') THEN
INSERT INTO proxy_audit (timestamp, elapsed, source, status, size,
method, url, domain, identity, mimetype)
VALUES(''epoch''::TIMESTAMP + (split_part(split_part(new.message, '' '', 1), ''.'', 1)::INT) * ''1 second''::INTERVAL,
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 1)::INT,
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 2),
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 3)::VARCHAR(15),
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 4)::INT,
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 5)::VARCHAR(15),
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 6)::VARCHAR(128),
split_part(split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 6), ''/'', 3)::VARCHAR(60),
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 7)::VARCHAR(25),
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 9));
END IF;
RETURN new;
END
' LANGUAGE plpgsql;
CREATE TRIGGER t_proxy_audit AFTER INSERT
ON facility_local4 FOR EACH ROW
EXECUTE PROCEDURE p_proxy_audit()
Once you get allot of data in your table, which happens quickly, you'll probably discover you need an index.
CREATE INDEX proxy_audit_i1 ON proxy_audit(identity, timestamp);
Now looking at a user's web usage is straight-forward. Finally.
2009-04-12
OBS Packages Updated
NOTE: The GroupDAV version 2 specification has not been published yet.
2009-03-12
Bug#394
To select database as your default project storage backend:
Defaults write NSGlobalDomain OGoDefaultProjectStorageBackend Database
To select the filesystem as your default project storage backend:
Defaults write NSGlobalDomain OGoDefaultProjectStorageBackend FileSystem
Most people I assume will want "Database". Remember that default values are CASE SENSITIVE!
If you (at least on r2181) attempt to create a project without selecting FileSystem/Database you'll get a popup warning: "Please specify your project storage!" If the default is set then the preferred storage backend is preselected (and can be changed).
Aside: If you use FileSystem projects make sure you have the SkyFSPath default set and that the referred to directory has the correct permissions.
2009-03-04
Packages Complete
Repositories at:
Initial testing on CentOS 5 seems pretty positive, after a:
$ yum install ogo-meta ogo-database-setup
Only glaring bug is that ogo-database-setup doesn't actually populate the database schema so that still needs to be done by hand:
$ cat pg-build-schema.psql | psql -h localhost -U OGo OGo
Then restarting the services gives you [as far as I can tell so far] a fully working OGo install.
P.S. Make sure your getting the most current packages from the repository. mod_ngobjweb in particular should be dated March 4th; with the previous version the web resources (icons, etc...) for the WebUI won't be found at the configured path.
2009-02-24
Setting the TimeZone in VMware's VIMA Appliance
[vi-admin@vima ~]$ date
Tue Feb 24 12:37:01 PST 2009
[vi-admin@vima ~]$ sudo rm /etc/localtime
[vi-admin@vima ~]$ sudo ln -s /usr/share/zoneinfo/America/Detroit /etc/localtime
[vi-admin@vima ~]$ date
Tue Feb 24 15:40:59 EST 2009
2009-01-04
Almost there
$ yum install ogo-meta
...
---> Package ogo-meta.i386 0:1.1-6.7 set to be updated
--> Processing Dependency: mod_ngobjweb for package: ogo-meta
--> Processing Dependency: ogo-environment for package: ogo-meta
---> Package postgresql.i386 0:8.1.11-1.el5_1.1 set to be updated
---> Package postgresql-libs.i386 0:8.1.11-1.el5_1.1 set to be updated
--> Finished Dependency Resolution
Error: Missing Dependency: ogo-environment is needed by package ogo-meta
Error: Missing Dependency: mod_ngobjweb is needed by package ogo-meta
2009-01-01
OpenGroupware Package Repositories
Not everything working yet, but getting there.
2008-09-21
OpenGroupware r2150 & RSS Feeds
- projectActions - Reports actions on tasks assigned to projects of which the user is a member either directly or via team membership.
- http://opengroupware.mormail.com/zidestore/so/${USER}/Tasks/delegated-actions-rss
- toDoActions - Reports actions on tasks of which the user is an executor either directly or via team membership.
- http://opengroupware.mormail.com/zidestore/so/${USER}/Tasks/project-actions-rss
- delegatedActions - Reports actions on tasks created by the user.
- http://opengroupware.mormail.com/zidestore/so/${USER}/Tasks/todo-actions-rss