Friday, November 2, 2007

Dummies Guide to Darwin Calendar Server

I just realized a long-time dream of synchronizing iCal across multiple computers without a .Mac account, and so I thought I'd share the joy with anyone else who is interested.

Mac OS X 10.5 Leopard includes a new version of iCal that can connect to a CalDAV server, and so it's simply a matter of setting up a CalDAV server. Leopard Server includes iCal Server, which I'm sure works just fine, but we haven't upgraded our servers, and I don't know when (or if) we're planning to do so. However, iCal Server is available in an open source version, Darwin Calendar Server, which doesn't include the Graphic Interface that I'm sure iCal Server includes.

Turns out that Darwin Calendar Server can be installed onto a Mac running Leopard (not necessarily Leopard Server) by a somewhat ordinary Joe like me without too much fuss. Here's the skinny:

1. The first step is downloading the Calendar Server source, which is actually accomplished using Subversion, which is installed with the Leopard Developer Tools. (You can also get Subversion elsewhere, as mentioned on Mac OS Forge.) Once you have Subversion installed, open the Terminal, and type the following:

svn checkout http://svn.calendarserver.org/repository/calendarserver/CalendarServer/trunk /Library/CalendarServer

This can apparently be done using XCode (also part of Leopard's Developer Tools) but it wasn't immediately obvious to me, since I'm not familiar with XCode.

2. Building the application from the source code is mercifully easy: move to the CalendarServer directory (cd /Library/CalendarServer) and run the following:
./run -s
The -s flag causes it to download the necessary supporting software and set up the service; this takes a little while, but it is verbose in its description, so it will be obvious when the process is complete.

3. Next, you'll need to create a configuration file, which involves making a copy of the included configuration template:
cp ./conf/caldavd-test.plist ./conf/caldavd-dev.plist

4. This is the part that I was stuck on for a while: in order to access the server from another computer (which is the whole point of doing this), you need to edit this configuration file you just created (/Library/CalendarServer/conf/caldavd-dev.plist). In that file, find the BindAddresses entry and change the 127.0.0.1 to the IP address or DNS name of the machine you're on. You can do this with Leopard's Property List Editor, as shown below.


5. Next, read through the QuickStart page on Mac OS Forge regarding directory services... I just used the XMLDirectoryService, which is on by default; you might want to edit /Library/CalendarServer/conf/accounts-test.xml to add a login for yourself.

6. You can then test things out by running the server:

./run

Open iCal and go to the Accounts pane of the Preferences window. Add a new account and fill out the login information you added to the accounts-test.xml file. Click the disclosure arrow, and enter your server address followed by ":8008" (i.e., myserver.com:8008) and leave the Kerberos checkbox unchecked. It should connect to the server and supply you with a default calendar (more can be added using the File > New Calendar submenu).

7. If things are working, you can quit the server by pressing Control-C in the terminal and run it as a background process by entering the following:

./run -d

This will run fine, but you'd have to remember to run it each time you restart the machine, so it's better to make a startup item. There is a bit of a trick to this, because it requires fixing a little bug in the "run" script, but here's how to do it:

8. To simplify the process of creating a launchd script (which is what will launch the server at system startup for you), download the latest OS X build of Lingon. Once you have downloaded it, open it up.

9. Click the "New" button to create a new service, and select "User Daemon" in the dialog that appears. Then enter the following information:
Name: com.apple.calendarserver
What: /Library/CalendarServer/run -d
When: Select "Run it when it is loaded by the system (at startup or login)"

10. Click "Save." Now for the nitty-gritty part:

11. Navigate to /Library/CalendarServer/ and make a backup copy of the "run" script. Open the original in a text editor (I used TextEdit) and search for these lines (they're toward the bottom of the script):

case "${USER}" in
  wsanchez)
    proto="svn+ssh";
    ;;
  *)
  proto="svn";
    ;;
esac;
svn_uri="${proto}://svn.twistedmatrix.com/svn/Twisted/branches/dav-acl-1608-4";
svn_get "Twisted" "${twisted}" "${svn_uri}" 19773;

Remove the necessary lines so that only these remain:

proto="svn";
svn_uri="${proto}://svn.twistedmatrix.com/svn/Twisted/branches/dav-acl-1608-4";
svn_get "Twisted" "${twisted}" "${svn_uri}" 19773;

Then find these:

case "${USER}" in
  cyrusdaboo)
    base="svn+ssh://cdaboo@svn.osafoundation.org/svn";
    ;;
  *)
  base="http://svn.osafoundation.org";
    ;;
esac;
svn_uri="${base}/vobject/branches/users/cdaboo/vavailability-173";
svn_get "vObject" "${vobject}" "${svn_uri}" 178;
put_setuptools "${vobject}";

And replace them with these:

base="http://svn.osafoundation.org";
svn_uri="${base}/vobject/branches/users/cdaboo/vavailability-173";
svn_get "vObject" "${vobject}" "${svn_uri}" 178;
put_setuptools "${vobject}";

12. Save your modified script and restart. iCal should now see the server running in the background.

This setup seems to have worked for me, and I've moved all my events from my regular iCal calendars into calendars that reside on my server (which can be done by dragging them to the new calendar in the Calendar list on the left). To do items are also managed this way, and the CalDAV server even synchronizes colors of calendars from machine to machine.

16 comments:

Reb said...

Dear Toby,

the tutorial is very detailed but following all your suggestions I am not able to run as a startup service. If I run it from terminal it is ok but even putting directly your startup files in my Leopard 10.5.2 the calendar server doesn't start automatically. In the console I get the following errors

com.apple.SystemStarter[28] /Library/CalendarServer/run: line 524: USER: unbound variable

could you help me

Thanks

Reb

Toby Rush said...

Reb, I'm not in a position to test this right now, but maybe you can try something for me to see if it fixes the problem: in the CalendarServer folder you downloaded from my site, open the "StartupParameters.plist" file in a text editor. Change the line:

OrderPreference = "None";

to

OrderPreference = "Late";

and see if that solves it. My thought is that it is looking for a variable that the system has not yet initialized, and perhaps by having the startup item load later, that variable will have been initialized.

I can also get it to work by creating a script (which I put here) which executes the necessary command and having it run at login, but that's not as cool as having it run on startup, especially for multi-user machines.

Let me know if the orderpreference trick works, and I'll take a look at this when I get a chance. Thanks!

Reb said...

Dear Toby,

thanks a lot for the fast answer. Unfortunately this doesn't help and the error remains the same as you see

1/10/08 10:02:08 AM com.apple.SystemStarter[29] Starting Calendar Server

1/10/08 10:02:11 AM com.apple.SystemStarter[29] /Library/CalendarServer/run: line 524: USER: unbound variable

Cheers
Reb

Toby Rush said...

Reb, I've updated the post with info on fixing the problem. It works for me... let me know if this solves it for you as well.

Leevi Graham said...

Hi Toby,

Thanks for your great tutorial. I setup the server and it runs fine on localhost (127.0.0.1) but when I ret and change it to my static IP (220.233.238.103) there server will not start.

I receive the following error: twisted.internet.error.CannotListenError: Couldn't listen on 220.233.238.103:8008: (49, "Can't assign requested address").

I have setup port forwarding on my router but when I ue the network monitor and scan for ports it is not open.

Any ideas?

Toby Rush said...

Leevi, this is getting to the edge of what I understand about networking, but I'll take a stab at it...

You say that you have a router with port forwarding, so I'm assuming that the static IP you're using is pointing to your router and that your computer(s) have internal IP addresses (e.g., 10.x.x.x). Is that right?

If so, I wonder if you need to set BindAddress in step 4 of the tutorial to that internal address instead of your static IP address. If you have your router doing port forwarding to send port 8008 traffic to the computer in question, then you should use the static IP address in iCal, since the router will send incoming traffic the right direction.

In other words, the CalendarServer needs to bind to the 10.x.x.x address because as far as it's concerned, that's the actual IP address of the computer. As far as clients elsewhere are concerned, though, they would use the 220.x.x.x address, because they aren't necessarily on the internal network.

Does that make sense? Or does it even help?

Leevi Graham said...

An I'm such an idiot :) Of course it makes sense.

However when I add my local ip (192.168.1.2) to the BindAddress the server automatically loads 127.0.0.1

I tested this by changing the BindAdress to something else and it tried to connect to the updated IP

Thanks for your help on the first part

Leevi Graham said...

Oh and the other thing that's a bit strange is that sometimes the server starts up with port 8009 or 8010

Leevi Graham said...

Thinking about it ... the sever loading 127.0.0.1 makes sense as its the same as 192.168.1.2 on my local machine...

If the router is sending traffic to my internal IP it should be ok...

testing now

Toby Rush said...

True... and if that doesn't work, you might try playing with the ServerHostName field, which is set to "localhost" by default. The comments in the caldavd-dev.plist file (which you can see if you open it in a text editor) mention that ServerHostName is what the server reports to outside addresses, so it might need to be set to your static IP address.

Sorry I'm not able to test in a router situation; let me know if you make any progress and I'd be happy to brainstorm with you!

JMK said...

Hi Toby,

No problems installing the server. And it runs fine as a normal user. However, I cannot run the server as root:

[memcached] can't run as root without the -u switch

appears in my log and the server doesn't run. This makes launching from the LaunchDaemon difficult. Any ideas?

Toby Rush said...

JMK, I'm not absolutely sure what's happening on your system, but it appears that the error you are seeing is related to memcached (http://www.danga.com/memcached/) which is a separate daemon that may or may not be part of the issue with your CalDAV server.

I know nothing about memcached, but if you are somehow using it to launch the CalDAV daemon, then my guess is that the -u flag allows you to indicate which user to use, and perhaps you can use the flag to specify the root user.

I used Lingon to handle launching CalDAV as a daemon. If you are also using Lingon and you are seeing this error message, then my assumption is that memcached is somehow interfering with the process. It may be worth researching memcached a little more to see what might be happening.

I'm sorry I can't be more than help than that; the issue is running up against the limits of my knowledge (which is a remarkably easy thing to do).

--Toby

daniel said...

Hi Toby,

Many thanks for your time to publish this article and your responses to posts. We have a CalDAV server running on Mac OS X 10.4 (i.e. not Leopard). It was set up by someone who no longer works in our office so I'm not entirely sure how it was set up, but it seems to follow your exact description. The only problem is the shared calendar often has weird behavior... for example, some recurring events that I posted on our shared/group calendar show me as the organizer, but I can only delete one event at a time (not the entire series of events) and I get the "Delete and Notify [the organizer]?" confirmation box when I delete... I'm the organizer so I shouldn't be notified, and I don't receive any notification when I confirm deletion... If you have any ideas about how to fix this issue, I would be grateful, but my real question is about moving the iCal server to a different box. I'm thinking I will upgrade the box to OS X 10.5 or Leopard Server (and run Apple's full iCal server). However, I don't know which files (databases?) to backup. If you can point me in the right direction to migrate, let me know. Many thanks.

Daniel.

StrawHousePig said...

I've got the memcached problem, too. To get it to run as root, look for these lines in twistedcaldav/config.py

"Memcached": {
"MaxClients": 5,

Alter it to...

"Memcached": {
"User": "root",
"MaxClients": 5,

Now edit twistedcaldav/cluster.py. Find...

config.Memcached["memcached"],
'-p', str(config.Memcached["Port"]),

Alter to...

config.Memcached["memcached"],
'-u', str(config.Memcached["User"]),
'-p', str(config.Memcached["Port"]),

Would be nice to not have to force it to run as root, but this should get it going with launchd.

t.hendricks said...

The Post of StrawHousePig helped me at least solving it, but they changed code:

The first change now has to take place in

twistedcaldav/stdconfig.py

The second went to
calendarserver/tap/caldav.py
and is now

config.Memcached.memcached,
"-p", str(config.Memcached.Port),
"-u", str(config.Memcached.User),

jsahiri said...

Hello Toby,
I followed the instructions on your very clear guide to Darwin Calendar Server. Everything went smoothly until I finished configuring the accounts-test.xml and did ./run. Then I got the error below:

2011-03-19 10:33:38-0400 [-] raise CannotListenError, (self.interface, self.port, le)
2011-03-19 10:33:38-0400 [-] twisted.internet.error.CannotListenError: Couldn't listen on 1:8008: [Errno 8] nodename nor servname provided, or not known.
launchctl bsexec failed: No such file or directory