Refactoring ark into "build", feedback needed


#1

Ark is getting a bit too hairy as I try to stuff more and more
functionality into it. The current Ark cookbook also has issues w/
idempotency that need to be addressed. I am thinking about creating a
new LWRP “build” for which ark is just a component. Tks to erik
hollensbe for suggesting the name “build”. the main idea is that build
has sub-resources like configure, make, python, ant, etc. that each
take their block

Here is the DSL that I have in mind. If this is a frankenstein, pls tell me

build “collectd”

this would call the ark provider in the build provider

ark do
url "http://…"
checksum "asdfasdf"
end
configure do # won’t run if “config.status” file exists
flags [ “–enable-python”, “–enable-jvm” ]
prefix "/opt"
end
make do
env [ “OS=linux” ]
creates 'bin/collectd’
end
make “install” do
creates "/usr/sbin/collectd"
end
end

easier to extend and more composable, you could even use it w/out ark
to grab and deposit the files and build could be extended to use
additional build tools like ant, cmake, scons, etc.

bash “untar files” do
code <<EOF
wget http://example.com/foobar.tar.gz -O /tmp/foobar
tar xvzf /tmp/foobar.tar.gz -C /opt
EOF
end

build “foobar” do
cmake do
flags [ “–with-cracklib”, “–with-postgres” ]
creates "bin/foobar"
end
cmake “install” do
creates "bin/foobar"
end
end

end

one of the downsides is that this will require a good amount of work
on my side, something I was hoping to avoid

I also need to balance this new dsl w/ the efficacy of the current ark
lwrp, which is really succinct for simple tasks. Perhaps I could do
this by leaving ark as a separate class but then creating a subclass
in Build::Ark. just an idea

feedback would be most appreciated!


#2

That looks like it may be a beautiful thing, especially if it can be
easily leveraged to do installs of libraries into the omnibus location
for things like, for example, the pg gem. It would probably be nice to
leave ark available as a standalone resource for things that don’t have
a build step… it doesn’t make a lot of sense to use a “build” resource
to download and unzip an archive.

Greg

On 11/01/2012 09:41 AM, Bryan Berry wrote:

Ark is getting a bit too hairy as I try to stuff more and more
functionality into it. The current Ark cookbook also has issues w/
idempotency that need to be addressed. I am thinking about creating a
new LWRP “build” for which ark is just a component. Tks to erik
hollensbe for suggesting the name “build”. the main idea is that build
has sub-resources like configure, make, python, ant, etc. that each
take their block

Here is the DSL that I have in mind. If this is a frankenstein, pls tell me

build “collectd”

this would call the ark provider in the build provider

ark do
url "http://…"
checksum "asdfasdf"
end
configure do # won’t run if “config.status” file exists
flags [ “–enable-python”, “–enable-jvm” ]
prefix "/opt"
end
make do
env [ “OS=linux” ]
creates 'bin/collectd’
end
make “install” do
creates "/usr/sbin/collectd"
end
end

easier to extend and more composable, you could even use it w/out ark
to grab and deposit the files and build could be extended to use
additional build tools like ant, cmake, scons, etc.

bash “untar files” do
code <<EOF
wget http://example.com/foobar.tar.gz -O /tmp/foobar
tar xvzf /tmp/foobar.tar.gz -C /opt
EOF
end

build “foobar” do
cmake do
flags [ “–with-cracklib”, “–with-postgres” ]
creates "bin/foobar"
end
cmake “install” do
creates "bin/foobar"
end
end

end

one of the downsides is that this will require a good amount of work
on my side, something I was hoping to avoid

I also need to balance this new dsl w/ the efficacy of the current ark
lwrp, which is really succinct for simple tasks. Perhaps I could do
this by leaving ark as a separate class but then creating a subclass
in Build::Ark. just an idea

feedback would be most appreciated!


#3

tks for your feedback Greg. I do intend to leave ark functioning as it
currently does for simple download-unpack-link tasks

the chef_gem already installs libraries into the omnibus location. I
use that to install the pg gem

On Thu, Nov 1, 2012 at 4:39 PM, Greg Symons gsymons@drillinginfo.com wrote:

That looks like it may be a beautiful thing, especially if it can be easily
leveraged to do installs of libraries into the omnibus location for things
like, for example, the pg gem. It would probably be nice to leave ark
available as a standalone resource for things that don’t have a build
step… it doesn’t make a lot of sense to use a “build” resource to download
and unzip an archive.

Greg

On 11/01/2012 09:41 AM, Bryan Berry wrote:

Ark is getting a bit too hairy as I try to stuff more and more
functionality into it. The current Ark cookbook also has issues w/
idempotency that need to be addressed. I am thinking about creating a
new LWRP “build” for which ark is just a component. Tks to erik
hollensbe for suggesting the name “build”. the main idea is that build
has sub-resources like configure, make, python, ant, etc. that each
take their block

Here is the DSL that I have in mind. If this is a frankenstein, pls tell
me

build “collectd”

this would call the ark provider in the build provider

ark do
url "http://…"
checksum "asdfasdf"
end
configure do # won’t run if “config.status” file exists
flags [ “–enable-python”, “–enable-jvm” ]
prefix "/opt"
end
make do
env [ “OS=linux” ]
creates 'bin/collectd’
end
make “install” do
creates "/usr/sbin/collectd"
end
end

easier to extend and more composable, you could even use it w/out ark
to grab and deposit the files and build could be extended to use
additional build tools like ant, cmake, scons, etc.

bash “untar files” do
code <<EOF
wget http://example.com/foobar.tar.gz -O /tmp/foobar
tar xvzf /tmp/foobar.tar.gz -C /opt
EOF
end

build “foobar” do
cmake do
flags [ “–with-cracklib”, “–with-postgres” ]
creates "bin/foobar"
end
cmake “install” do
creates "bin/foobar"
end
end

end

one of the downsides is that this will require a good amount of work
on my side, something I was hoping to avoid

I also need to balance this new dsl w/ the efficacy of the current ark
lwrp, which is really succinct for simple tasks. Perhaps I could do
this by leaving ark as a separate class but then creating a subclass
in Build::Ark. just an idea

feedback would be most appreciated!


#4

You’re not running into the problem of conflicting OpenSSL versions
(i.e. COOK-1406)? What I meant was installing a different libpsql for
omnibus so using chef_gem will work if the system OpenSSL or other
libraries are not ABI compatible with the versions in the omnibus install.

Greg

On 11/01/2012 11:00 AM, Bryan Berry wrote:

tks for your feedback Greg. I do intend to leave ark functioning as it
currently does for simple download-unpack-link tasks

the chef_gem already installs libraries into the omnibus location. I
use that to install the pg gem

On Thu, Nov 1, 2012 at 4:39 PM, Greg Symons gsymons@drillinginfo.com wrote:

That looks like it may be a beautiful thing, especially if it can be easily
leveraged to do installs of libraries into the omnibus location for things
like, for example, the pg gem. It would probably be nice to leave ark
available as a standalone resource for things that don’t have a build
step… it doesn’t make a lot of sense to use a “build” resource to download
and unzip an archive.

Greg

On 11/01/2012 09:41 AM, Bryan Berry wrote:

Ark is getting a bit too hairy as I try to stuff more and more
functionality into it. The current Ark cookbook also has issues w/
idempotency that need to be addressed. I am thinking about creating a
new LWRP “build” for which ark is just a component. Tks to erik
hollensbe for suggesting the name “build”. the main idea is that build
has sub-resources like configure, make, python, ant, etc. that each
take their block

Here is the DSL that I have in mind. If this is a frankenstein, pls tell
me

build “collectd”
# this would call the ark provider in the build provider
ark do
url "http://…"
checksum "asdfasdf"
end
configure do # won’t run if “config.status” file exists
flags [ “–enable-python”, “–enable-jvm” ]
prefix "/opt"
end
make do
env [ “OS=linux” ]
creates 'bin/collectd’
end
make “install” do
creates "/usr/sbin/collectd"
end
end

easier to extend and more composable, you could even use it w/out ark
to grab and deposit the files and build could be extended to use
additional build tools like ant, cmake, scons, etc.

bash “untar files” do
code <<EOF
wget http://example.com/foobar.tar.gz -O /tmp/foobar
tar xvzf /tmp/foobar.tar.gz -C /opt
EOF
end

build “foobar” do
cmake do
flags [ “–with-cracklib”, “–with-postgres” ]
creates "bin/foobar"
end
cmake “install” do
creates "bin/foobar"
end
end

end

one of the downsides is that this will require a good amount of work
on my side, something I was hoping to avoid

I also need to balance this new dsl w/ the efficacy of the current ark
lwrp, which is really succinct for simple tasks. Perhaps I could do
this by leaving ark as a separate class but then creating a subclass
in Build::Ark. just an idea

feedback would be most appreciated!


#5

so far i have not run into that problem on Centos 6.2, 6.3. can’t
speak for other distros

On Thu, Nov 1, 2012 at 5:27 PM, Greg Symons gsymons@drillinginfo.com wrote:

You’re not running into the problem of conflicting OpenSSL versions (i.e.
COOK-1406)? What I meant was installing a different libpsql for omnibus so
using chef_gem will work if the system OpenSSL or other libraries are not
ABI compatible with the versions in the omnibus install.

Greg

On 11/01/2012 11:00 AM, Bryan Berry wrote:

tks for your feedback Greg. I do intend to leave ark functioning as it
currently does for simple download-unpack-link tasks

the chef_gem already installs libraries into the omnibus location. I
use that to install the pg gem

On Thu, Nov 1, 2012 at 4:39 PM, Greg Symons gsymons@drillinginfo.com
wrote:

That looks like it may be a beautiful thing, especially if it can be
easily
leveraged to do installs of libraries into the omnibus location for
things
like, for example, the pg gem. It would probably be nice to leave ark
available as a standalone resource for things that don’t have a build
step… it doesn’t make a lot of sense to use a “build” resource to
download
and unzip an archive.

Greg

On 11/01/2012 09:41 AM, Bryan Berry wrote:

Ark is getting a bit too hairy as I try to stuff more and more
functionality into it. The current Ark cookbook also has issues w/
idempotency that need to be addressed. I am thinking about creating a
new LWRP “build” for which ark is just a component. Tks to erik
hollensbe for suggesting the name “build”. the main idea is that build
has sub-resources like configure, make, python, ant, etc. that each
take their block

Here is the DSL that I have in mind. If this is a frankenstein, pls tell
me

build “collectd”
# this would call the ark provider in the build provider
ark do
url "http://…"
checksum "asdfasdf"
end
configure do # won’t run if “config.status” file exists
flags [ “–enable-python”, “–enable-jvm” ]
prefix "/opt"
end
make do
env [ “OS=linux” ]
creates 'bin/collectd’
end
make “install” do
creates "/usr/sbin/collectd"
end
end

easier to extend and more composable, you could even use it w/out ark
to grab and deposit the files and build could be extended to use
additional build tools like ant, cmake, scons, etc.

bash “untar files” do
code <<EOF
wget http://example.com/foobar.tar.gz -O /tmp/foobar
tar xvzf /tmp/foobar.tar.gz -C /opt
EOF
end

build “foobar” do
cmake do
flags [ “–with-cracklib”, “–with-postgres” ]
creates "bin/foobar"
end
cmake “install” do
creates "bin/foobar"
end
end

end

one of the downsides is that this will require a good amount of work
on my side, something I was hoping to avoid

I also need to balance this new dsl w/ the efficacy of the current ark
lwrp, which is really succinct for simple tasks. Perhaps I could do
this by leaving ark as a separate class but then creating a subclass
in Build::Ark. just an idea

feedback would be most appreciated!


#6

That would make sense, since it looks like it’s a problem on Ubuntu
12.04. Joshua Timberman has a workaround at
https://gist.github.com/3954641, but a more general solution to omnibus
library conflicts would be a little nicer.

Greg

On 11/01/2012 11:30 AM, Bryan Berry wrote:

so far i have not run into that problem on Centos 6.2, 6.3. can’t
speak for other distros

On Thu, Nov 1, 2012 at 5:27 PM, Greg Symons gsymons@drillinginfo.com wrote:

You’re not running into the problem of conflicting OpenSSL versions (i.e.
COOK-1406)? What I meant was installing a different libpsql for omnibus so
using chef_gem will work if the system OpenSSL or other libraries are not
ABI compatible with the versions in the omnibus install.

Greg

On 11/01/2012 11:00 AM, Bryan Berry wrote:

tks for your feedback Greg. I do intend to leave ark functioning as it
currently does for simple download-unpack-link tasks

the chef_gem already installs libraries into the omnibus location. I
use that to install the pg gem

On Thu, Nov 1, 2012 at 4:39 PM, Greg Symons gsymons@drillinginfo.com
wrote:

That looks like it may be a beautiful thing, especially if it can be
easily
leveraged to do installs of libraries into the omnibus location for
things
like, for example, the pg gem. It would probably be nice to leave ark
available as a standalone resource for things that don’t have a build
step… it doesn’t make a lot of sense to use a “build” resource to
download
and unzip an archive.

Greg

On 11/01/2012 09:41 AM, Bryan Berry wrote:

Ark is getting a bit too hairy as I try to stuff more and more
functionality into it. The current Ark cookbook also has issues w/
idempotency that need to be addressed. I am thinking about creating a
new LWRP “build” for which ark is just a component. Tks to erik
hollensbe for suggesting the name “build”. the main idea is that build
has sub-resources like configure, make, python, ant, etc. that each
take their block

Here is the DSL that I have in mind. If this is a frankenstein, pls tell
me

build “collectd”
# this would call the ark provider in the build provider
ark do
url "http://…"
checksum "asdfasdf"
end
configure do # won’t run if “config.status” file exists
flags [ “–enable-python”, “–enable-jvm” ]
prefix "/opt"
end
make do
env [ “OS=linux” ]
creates 'bin/collectd’
end
make “install” do
creates "/usr/sbin/collectd"
end
end

easier to extend and more composable, you could even use it w/out ark
to grab and deposit the files and build could be extended to use
additional build tools like ant, cmake, scons, etc.

bash “untar files” do
code <<EOF
wget http://example.com/foobar.tar.gz -O /tmp/foobar
tar xvzf /tmp/foobar.tar.gz -C /opt
EOF
end

build “foobar” do
cmake do
flags [ “–with-cracklib”, “–with-postgres” ]
creates "bin/foobar"
end
cmake “install” do
creates "bin/foobar"
end
end

end

one of the downsides is that this will require a good amount of work
on my side, something I was hoping to avoid

I also need to balance this new dsl w/ the efficacy of the current ark
lwrp, which is really succinct for simple tasks. Perhaps I could do
this by leaving ark as a separate class but then creating a subclass
in Build::Ark. just an idea

feedback would be most appreciated!