Python XML-RPC Presentation

Presented on using XML-RPC with Python to the Grand Rapids Python Users Group (GRPUG).

Presentation File

While it is admittedly a runt of an RPC solution I've found XML-RPC to be very useful and have used it extensively in both my work on OpenGroupware and other projects.

Logging *USEFUL* Web Access To PostgreSQL

It is pretty easy to configure syslog to log system messages to a PostgreSQL database; and that sure beats flat files. With some GRANT/REVOKE goodness this setup can be made quite secure - only allowing the message import process to insert messages and denying the ability to everyone else to modify the contents of the database. But all that data is still pretty ugly to deal with as the messages are just long strings; so some method is needed to break down that "data" into "information". I had the specific need to do this with SQUID weblogs. I already had syslog logging to a PostgreSQL database so the first step was just to have SQUID use syslog to log access. That is easily accomplished like:

cache_access_log syslog:LOG_LOCAL4

After a restart SQUID messages show up in the LOCAL4 facility (table: facility_local4) with the priority of "info". Then I created a table to hold the parsed messages:

CREATE TABLE proxy_audit (
timestamp TIMESTAMP,
elapsed INT,
source VARCHAR(40),
status VARCHAR(15),
size INT,
method VARCHAR(15),
url VARCHAR(128),
domain VARCHAR(60),
identity VARCHAR(25),
mimetype VARCHAR(45));

Last a trigger on the LOCAL4 log parses incoming messages into this table:

CREATE OR REPLACE FUNCTION p_proxy_audit() RETURNS trigger AS '
BEGIN
IF (new.hostname = ''hod-a'' AND new.priority = ''info'') THEN
INSERT INTO proxy_audit (timestamp, elapsed, source, status, size,
method, url, domain, identity, mimetype)
VALUES(''epoch''::TIMESTAMP + (split_part(split_part(new.message, '' '', 1), ''.'', 1)::INT) * ''1 second''::INTERVAL,
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 1)::INT,
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 2),
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 3)::VARCHAR(15),
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 4)::INT,
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 5)::VARCHAR(15),
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 6)::VARCHAR(128),
split_part(split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 6), ''/'', 3)::VARCHAR(60),
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 7)::VARCHAR(25),
split_part(TRIM(substring(new.message, strpos(new.message, '' ''))), '' '', 9));
END IF;
RETURN new;
END
' LANGUAGE plpgsql;
CREATE TRIGGER t_proxy_audit AFTER INSERT
ON facility_local4 FOR EACH ROW
EXECUTE PROCEDURE p_proxy_audit()

Once you get allot of data in your table, which happens quickly, you'll probably discover you need an index.

CREATE INDEX proxy_audit_i1 ON proxy_audit(identity, timestamp);

Now looking at a user's web usage is straight-forward. Finally.

OBS Packages Updated

OpenGroupware packages on the OBS have been updated to r2207. This revision provides improved GroupDAV version 1 compatibility and adds GroupDAV version 2 support. The upside of those improvements is that the new ZideOne Outlook plugin should work well with this revision (or later) of OpenGroupware. Numerous enhancements and bug fixes in Logic and zOGI are also included.

NOTE: The GroupDAV version 2 specification has not been published yet.

Bug#394

OpenGroupware revision 2181 closes Bug#394. It is now possible to set a default to determine the default storage backend for new projects. (Wow, with GNUstep based stuff one ends up using the term "default" so often in different ways....)

To select database as your default project storage backend:
Defaults write NSGlobalDomain OGoDefaultProjectStorageBackend Database

To select the filesystem as your default project storage backend:
Defaults write NSGlobalDomain OGoDefaultProjectStorageBackend FileSystem

Most people I assume will want "Database". Remember that default values are CASE SENSITIVE!

If you (at least on r2181) attempt to create a project without selecting FileSystem/Database you'll get a popup warning: "Please specify your project storage!" If the default is set then the preferred storage backend is preselected (and can be changed).

Aside: If you use FileSystem projects make sure you have the SkyFSPath default set and that the referred to directory has the correct permissions.

Packages Complete

OpenGroupware packages have been completely built on the build service for CentOS5, Fedora 9, openSUSE 10.3, RHEL 5, and SLES10. The Fedora 10 and openSUSE 11.x packages needs some additional work.

Repositories at:

• CentOS 5
• Fedora 9
• Fedora 10
• openSUSE 10.3
• openSUSE 11.0
• openSUSE 11.1
• RHEL 5
• SLES 10

Initial testing on CentOS 5 seems pretty positive, after a:
$ yum install ogo-meta ogo-database-setup

Only glaring bug is that ogo-database-setup doesn't actually populate the database schema so that still needs to be done by hand:

$ cat pg-build-schema.psql | psql -h localhost -U OGo OGo

Then restarting the services gives you [as far as I can tell so far] a fully working OGo install.

P.S. Make sure your getting the most current packages from the repository. mod_ngobjweb in particular should be dated March 4th; with the previous version the web resources (icons, etc...) for the WebUI won't be found at the configured path.

Setting the TimeZone in VMware's VIMA Appliance

After importing the VIMA appliance the time zone is Pacific time. When going to change the time zone you'll discover that they include neither redhat-config-date or timeconfig. So one has to go about it old-school:

[vi-admin@vima ~]$ date
Tue Feb 24 12:37:01 PST 2009
[vi-admin@vima ~]$ sudo rm /etc/localtime
[vi-admin@vima ~]$ sudo ln -s /usr/share/zoneinfo/America/Detroit /etc/localtime
[vi-admin@vima ~]$ date
Tue Feb 24 15:40:59 EST 2009

Almost there

The ogo-meta package now almost installs on CentOS5 except that we don't have the ogo-environment or the ngobjweb packages. OpenSUSE 11.0 and 11.1 have some problems building anything beyond ogo-gnustep_make; OpenSUSE 10.3 builds up to the same point as CentOS5.


$ yum install ogo-meta
...
---> Package ogo-meta.i386 0:1.1-6.7 set to be updated
--> Processing Dependency: mod_ngobjweb for package: ogo-meta
--> Processing Dependency: ogo-environment for package: ogo-meta
---> Package postgresql.i386 0:8.1.11-1.el5_1.1 set to be updated
---> Package postgresql-libs.i386 0:8.1.11-1.el5_1.1 set to be updated
--> Finished Dependency Resolution
Error: Missing Dependency: ogo-environment is needed by package ogo-meta
Error: Missing Dependency: mod_ngobjweb is needed by package ogo-meta


OpenGroupware Package Repositories

Currently working on getting OpenGroupware (and related GNUStep-make, SOPE, & ngobjweb) packages built of the Novell Build Service. Repositories at:

• CentOS 5
• Fedora 9
• openSUSE 10.3
• openSUSE 11.0
• openSUSE 11.1
• RHEL 5
• SLES 10

Not everything working yet, but getting there.

OpenGroupware r2150 & RSS Feeds

As of r2150 OpenGroupware now supports the following RSS 2.0 feeds:

• projectActions - Reports actions on tasks assigned to projects of which the user is a member either directly or via team membership.
• http://opengroupware.mormail.com/zidestore/so/${USER}/Tasks/delegated-actions-rss
• toDoActions - Reports actions on tasks of which the user is an executor either directly or via team membership.
• http://opengroupware.mormail.com/zidestore/so/${USER}/Tasks/project-actions-rss
• delegatedActions - Reports actions on tasks created by the user.
• http://opengroupware.mormail.com/zidestore/so/${USER}/Tasks/todo-actions-rss

By default the 150 most recent task actions are included in the feed; this value can be adjusted by including a "limit" value in the URL.

getAuditEntries added to zOGI r2095

The getAuditEntries method was added to the zOGI API as of r920, and added to the ZideStore trunk in r2095. getAuditEntries provides the ability to retrieve the audit entries from the server's database that have occurred since a specified entry. Using this feature a service can page through server changes and synchronize some repository; this allows functionality equivalent to that provided by MOGIMon but without a back-door database connection. Since audit records are serialized with integer ids in the OpenGroupware database this acts very much like the uSNChanged attribute provided by Microsoft Active Directory.

I've uploaded new OpenGroupware RPMs for openSUSE 10.2. These include the recent improvements in calendar permissions mentioned here. The packages can be downloaded from the OpenGroupware.Org docs plone here.

openSUSE 10.2 & gcc-objc

Neither the openSUSE 10.2 CDs nor the DVD image contain the the gcc-obj or the libobjc packages required to compile and run Objective-C applications such as OpenGroupware. I'm led to believe that the commercial DVD does contain more packages as it is multi-layered.

Fortunately this oversight is easily remedied; you just need to add the online package repository as an installation source. Go into YaST / Software / Installation Source, then add an HTTP source with a "Server Name" of "download.opensuse.org/distribution/10.2/repo/oss/". This is the default package repository for all Open Source packages provided with openSUSE 10.2. Clicking "Next" will then cause YaST to thrash around for awhile as it downloads what it needs to support the installation source. After prompting for acceptance of the same license agreement you accepted when installing the distribution the channel should be available.

Searching in Software / Software Management you should now find three Objective-C related packages: gcc41-objc, gcc-obj, and libobjc41. Only the latter is required to run Objective-C applications.

Once libobjc41 is installed then the OpenGroupware packages for OpenSuSE 10.0 will install and run.