Dreamcat4/brew-launchd For Mac
The ruby plist library is plist4r and has been in Stable 1.1 release for a while now. The Plist4r Github repo is also where those tests and the associated documentation are maintained. Brew is given a streamlined, embedded version of plist4r, specially tailored for Brew.
Example: (2). There is now a fully-functional extension for brew (called brew-launchd). You will need to checkout this side branch Then you can brew launchd -help or man brew-launchd for more info. Converted all the existing formula over. They're all in the topic branch.
I was able to figure it out. Thanks for the pointers. Here are my notes, in case anyone else is interested: First, I found the.plist files for each database, then I sudo cp'd them into /Library/LaunchDaemons/ From there, MySQL was fine, CouchDB wanted to run as a non-existant 'couchdb' user, and Postgresql had a formatting error. So, I changed CouchDB to run as my normal user, because I'm not picky about that sort of thing. Then, I just had to fix the Postgres line, which wasn't closed right. It basically needed to have myusername on one line, but it was on two. I may put in a separate ticket about this.
So, I'm OK for the time being, but I wonder if a wiki page about launchd might be a good idea, at least. It is a good learning experience to have to figure this stuff out, but it's also frustrating!;). Hmm, using your gist as an example, can you see why we should have the Brew formulae to symlink those plists from the Cellar into $brewprefix/Library/LaunchDaemons? $ export brewprefix=/usr/local $ ls -ls $brewprefix/Library/LaunchDaemons com.mysql.mysqld.plist@ - /usr/local/Cellar/mysql/5.1.41/com.mysql.mysqld.plist org.apache.couchdb.plist@ - /usr/local/Cellar/couchdb/0.10.1/Library/LaunchDaemons/org.apache.couchdb.plist org.postgresql.postgres.plist@ - /usr/local/Cellar/postgresql/8.4.1/org.postgresql.postgres.plist Whats currently missing is a macro in the formulae.rb to identify which of the installed files are the launchd plist file(s). I guess we need an attribute in the Formula base class to hold an array of plist files. Then in your formulae you would put a line: @launchdplist = prefix+'org.postgresql.postgres.plist' or @launchdplists = prefix+'org.postgresql.postgres.plist', prefix+'org.postgresql.postgrehelper.plist' Please see gist.
Update: As of today, the main plist block syntax all implemented, including validation and inlined rdoc comments for all the more complex nested launchd features. The simple plist keys are also fully validated too.
That doesn't mean there are tests yet. It means that the user input used to make the plists will all be validated. This is core changes to the brew software, and requiring some reviewer feedback from mxcl. Please update the tags to 'core-change'. Generally the code is still a bit messy around the edges and whatnot. However all the major functinality (of the brew-core code) is now implemented.
All the tricky parts. So its now the 'shallow slope', tidying up, polishing, etc. Branch: One piece omitted the bit for reading in existing plists. That would have been cool to do, so you could override the parts of a packages' plist file that you wanted.
(rather than re-write a whole new plist). I've left that class commented out because it depends heavily upon libxml-ruby, which is a 3.5MB compiled ruby gem. I don't think that can be justified for Brew.
This plist reading class could be re-written in hpricot, EDITas is v.similar syntax to the existing code. Twist: unfortunately the Hpricot gem is written in C and therefore requires a compilation step.
So it is not 'ready-to-go' like other pure ruby code. Its much harder to justify doing this unless those dependencies are going to be accepted into brew-core. Adam has been experimenting in REXML recently, and has written a class for writing out the plist. Adams REXML code wasnt designed for reading in an existing plist.
But if we want that feature then its probably best thing to work with Adam + integrate his REXML work into the parser class here to get it done for brew-core. Otherwise we might omit just this 'plist editing' feature and regard it as non-essential. Other features and functionality (as discussed previously) are unaffected. Should be releasing a seperate ruby gem of this at some point in the future. That will have all of the same code we want to put into brew, and including the parts that depend upon libxml. Note: This code does not include the seperate plist starter / monitor program which is external to brew.
That one is to be developed seperately afterwards, and packaged as a brew formula. Thanks for the tip. The biggest problem I can see for RubyCocoa is that its not universally supported for all users. I agree its unnecessary to use standard xml parsing / construction techniques if RubyCocoa is available.
Its just that im not all sure that in all cases RubyCocoa is available. Fine if you live in a world where you think everyone uses the stock 10.6 ruby.
But many people don't. They'll use rvm, or install a newer version of Ruby from source.
These people are many. They may raise error and report bugs on brew when it isn't working for them.
I don't want to be responsible for that. I want to make a solution that will work universally. Also - did you know that compiling ruby with RubyCocoa needs '—enabled-shared', which supposedly slows down REE by 20%? As for my current implementation, Haml is embedded into Brew as a single.rb file.
Plists are generated with haml (which is a pretty decent xml ML). This works for everybody. As would REXML or ERB. Second point: Most of the harder-to-replace code is in the Ruby DSL and validation classes.
These are written for checking whether input conforms to std Ruby classes (Hash, Array, Fixnum, String, etc). Im just not sure yet if its possible to use RubyCocoa so interchangeably here, as RubyCocoa represents its data in native Cocoa classes. In other words, its possible that we might loose compatibility going back the other way. Can cause problems as this code must later gem-ified later for chef. So clearly I'd like to stand back a bit to see what Mxcl thinks of all this. If hes OK to accept an xml-based implementation, then i'm much happier to provide one.
Okay, after speaking with mxcl on IRC it turns out we can use RubyCocoa. To do so it must be kept wrapper within its own seperate process (spawned from brew).
We pipe all our data through STDIN / STDOUT. This will work for everyone, including those people who've installed their own version of ruby. RubyCocoa also provides these methods to do class conversion: nsdictionary.toruby OSX::NSDictionary.alloc.initWithDictionary(hash) So that puts to rest the other issue about data types and validation. With all these problems solved, i shall be able to rework my existing classes to use RubyCocoa and 0 new issues. Very pleased and its a win-win. RubyCocoa was shipped with both 10.5 and 10.6. Therefore, from a system perspective, RubyCocoa is universally supported for all users.
Homebrew should be relying on what's available in a systems context; its shebang comment should be pointing to the system Ruby. If necessary, the 'brew' Ruby script should be replaced with a shell script that always launches the 'brew' command-line with the system Ruby and resets the Ruby environment to force the system's Ruby for use. It should NOT be using an alternate Ruby that's installed by the user. In this case, I don't care what most users use for their primary Ruby; Homebrew shouldn't, either. It's a simple matter of what your baseline dependencies are. (This is why, for a set of scripts we use at work, we write them in /bin/sh—we KNOW how that behaves 100% of the time on 100% of our install targets and that it's always there.) There's a third point, but it's not completely germane to this particular discussion so I will message you privately. Okay, here is the wrapper code for RubyCocoa (running as a seperate process).
Dreamcast 4/brew-launchd For Mac Os
I used Popen4 from chef, which is just a standalone method and can be included as a support function in brew. Quite frankly, i wouldn't want to do this with brew's version of system, of which i have no intimate knowledge.
Hey, at least with Popen4 its trusted, reliable, and fully documented. As a bonus, popen4 returns the stderr and stdout are returned back to us as seperate streams. So I hope no-ones going to argue heavily against that decision.
If you do want popen changed for something else then fork the gist and change it yourself, be my guest. To try this gist: $ git clone git://gist.github.com/323060.git $ cd 323060 Put all of the gist files in the same directory. Cd to that folder, and load up irb. Then run interactively those lines in brew.rb / def test. Sorry, that these comment comes in a bit late. In the start of this issue mxcl mentions that he wants users to learn to use Homebrew, and have some knowledge. Why not split the auto starting of 'servers'/'daemons' (as macports does) into seperate recipes?
Example: mysql & mysql-server, couchdb & couchdb-server Where the.-server recipe depends on their 'parent' recipe. This could give the more advanced user the ability to have control over the installation process. For instance make the installation of the package and server a 2 phase process: (1) install the package without 'sudo' and (2) install only the '-server' part with sudo. (because only for the latter 'sudo' is required) Besides that, you now give additional control to the user, if they would like to autostart the things.
The user can't blame brew that it automatically installs and starts loading all kind of code into memory: they did it themselves and could more easily undo the auto starting by un-installing only the server component. This might also lead the different server recipes, generating different plists, that for instance: (a) listen to their specified networking ports and start upon first user request, or (b) that auto restart the daemon when it quits, etcetera.
Just to give an update on this. Things are taking a little longer than expected however this launchd work is still continuing as a high priority for me. Currently I am developing RSpec test suite for the Plist4r gem. Some tricky bits in relation to the backends test harness.
After thats solved then the rest should progress more easily. This is all assuming that Homebrew Project maintainers require this Plist ruby library to have regression tests, before accepting a patch of it. If anyone wants to work on some frontend RSpec tests (Plist4r gem), then perhaps that might overall speed up the ticket. But all fine otherwise, thank you & good day. Update: Have finished regression tests for Plist4r and released it as v1.1 rubygem. Next in my hombrew branch I have a script to the import and sync the Plist4r tree to the homebrew core code.
It makes syncing future updates of Plist4r very easy. Then I added the necessary hooks for the formulae to declare a plist file and converted some existing formulae over to work in the new way. You can see those proposed changes at the dreamcat4 repo, branch 'issue-148-launchd' The next thing is to write a special kind of background watcher program. Which is the part III (final part). Its already started the project under the name dreamcat4/blew. If you can give it a better working name than 'blew' - please let me know your suggestions. I am happy to move over to working on that project.
Later on perhaps the blew tool is finished, it can be discussed further. Oh, I just found out today that brew has a new feature called 'external commands'. So i guess that would be an appropriate place to call in from brew to the new launchd tool. That has already a naming convention.
I expect change the name of blew project to something else, and have it callable through either its own seperate command (as it can be a rubygem), but also possible to be exactly such external brew command as in the link above. For one example: You might run brew load mysql or brew start mysql to manually bring up the mysql service after installing it.
Or apache / nginx whatever. There might be an auto-start command, to bring services up automatically (once they are installed). Thats a very debian-ish way to do things. If i assign these tasks to specific command names, they will occupy the global brew namespace. Then you won't be able to run some other command happend to have the same name. So perhaps naming these commands should require a little bit of discussion. If you have anything to say about that, please let me know.
The thing is that ^^ git branch has the core changes for brew in it. You need to alter the brew core to make use of this extension.
The extension part doesnt work without the core changes merged into brew. Formula writers also need to declare their plists in the new ruby-native DSL. The core changes branch includes also the necessary example formulae, which have been converted to the new standard. But im all up with closing this ticket to open one or more new tickets. If that makes tracking these launchd additions any easier.
If you have any time to dedicate to this issue: follow the instructions above and just try the feature out. Get a feeling for it. And hopefully Max himself can try it out sometime too. See if its allright / meets expectations. Hi Iwe, I am delighted to hear more people taking an interest.
It sounds like you have some understanding of the code under review. Have you tried it? In terms of the main thrust of what you are suggesting.
Bear in mind that losing the core changes = raising the barrier for formula writers. However it sounds as if you are in favour of the extension part for managing installed services? Lastly (P.S.) you mention the idea for making a fallback to check if a hardcoded plist file already exists and if so to pick up the hardcoded file. That is indeed useful when the package has supplied / built its own plist file.This suggested functionality has already been implemented as a working feature of launchdplist. Just write in the formula launchdplist and the file already exists it will be loaded first. Then any selective changes you specify will apply ontop (for example to use a variable from your formula).
Those parts overwrite. Then it saves back to the plist file. If you specify no changes then the original file is kept as-is. It would be great to provide you guys with those remaining missing documentation for this new launchdplist directive.
Everything else has been documented. The correct place for that is probably homebrew wiki, once changes have been accepted. In the meantime I shall put together a blog post about it. Yes, I've tried it, thanks for the instructions. And it works like expected:) and I browsed through all the code. On the other hand, to be honest writing formulas should be as simple as possible yes, but launchd plist files are not something distributed within a.tar.gz of redis, but additional, user supplied stuff (in most cases) and as soon as it is user supplied I think the formula author should know what he's doing, so the additional complexity is acceptable (IMHO).
What I suggest is that instead of adding additional functionality to homebrew and introducing core changes - a lightweight layer which basically adds the same, but requires the authors to write/know more. Of course a plist DSL would be sweet, but is it worth all the additional code & dependencies? I don't know why, but these core changes et all, just don't feel like 'the-homebrew-way-of-doing-stuff' - don't get me wrong, this is just my opinion:). Yes, I've tried it, thanks for the instructions.
And it works like expected:) and I browsed through all the code. On the other hand, to be honest writing formulas should be as simple as possible yes, but launchd plist files are not something distributed within a.tar.gz of redis, but additional, user supplied stuff (in most cases) and as soon as it is user supplied I think the formula author should know what he's doing, so the additional complexity is acceptable (IMHO). What I suggest is that instead of adding additional functionality to homebrew and introducing core changes - a lightweight layer which basically adds the same, but requires the authors to write/know more. Of course a plist DSL would be sweet, but is it worth all the additional code & dependencies?
I don't know why, but these core changes et all, just don't feel like 'the-homebrew-way-of-doing-stuff' - don't get me wrong, this is just my opinion:). Well my original reply was just trashed with a github 404 error msg. To respond to one specific concern. The question posed was: 'Of course a plist DSL would be sweet, but is it worth all the additional code & dependencies?' Why do we have this plist DSL?
Its been backed by Jan 10th 2010 comments. See earlier in this thread. Mxcl and trevorturk were in favor of it. In terms of the implementation. To work with the distribution model for homebrew we dont want to have to install some extra gem or software packages along with homebrew. The plist DSL must be accessible from the formula level - so thats why its been inlined into the core. Simple as that.
I absolutely stand behind these plist classes as being the best available out there. Much care has been taken to ensure the quality and compatability of this code with the homebrew core code. Including checking that overloaded classes dont conflict - as homebrew has a habit of extending classes like Pathname, ARGV etc. Its also been through so many fucking tests. Lots of real-world plist data files.
Various auto-generated data sets. Performance / scalability testing.
Dreamcast 4/brew-launchd For Mac Os X
Full suite of regression tests upstream at dreamcat4/plist4r. Comprehensively documented inside and out. Trying to stay open-minded about this. But its kindda hard after writing so much of that bloody code!
$ xcode-select -install $ gem uninstall nokogiri $ gem install nokogiri $ rbenv rehash Here are a few important notes regarding this issue:. I use rbenv for managing rubies, which may change your process. You'll (likely) needed to have gone through the reinstallation of your Ruby. You'll also probably have to uninstall and reinstall the bundle for every Ruby project using this version of Ruby.
Postgres I had installed postgres via Homebrew and the upgrade to Yosemite removed some directories. Follow I was able to get postgres running again.