Hi I have wrote a previous topic about migrating data b/w chef servers, but it seems a whole bunch way is not good for me. What I need is more granular approach.
Given I have existed client, node, client key (pem) on chef11 server. Now I need to import all this into new chef12 server. Which is the best way to do this? Thanks.
I would go with the chef-server-ctl upgrade commands on the new server, you’ll need a chef11 admin key .
chef-server-ctl chef12-upgrade-download -s chef11.server.url -u <admin-user> -k </path/to/admin-user.pem> -d /tmp/chef11chef-server-ctl chef12-upgrade-data-transform -d /tmp/chef11 -e /tmp/chef12 # it will ask for org name, etcrm -rf /tmp/chef12/organizations/<your-org>/cookbooks/* # This is a optional step, if you wish to migrate I would keep the cookbooks.chef-server-ctl chef12-upgrade-data-upload -e /tmp/chef12 # again it will ask for org short name before uploadingThere’s others options for each command, call them with --help if you wish more control and less interactivity with them.
If you encounter errors while uploading cookbooks I’ve made a summary of what I did on my own migration on this repo
Hope this help
Thanks for your feedback @Tensibai , does chef12-upgrade-data-transform allow to migrate only a distinct objects, like only a single client, node? Unfortunately Migration in a whole bunch mode does not feet much for me …
And also, does chef12-upgrade-data-transform preserve existed client keys (pem files) ?
Thanks
I think I’ve forgot to mention this is mainly targeted to open source chef11 to chef12 (where you have to link toward an organization).
But yes the goal is to maintain node/client relationship and the ACL behind them inside chef12 server.
You may copy part of the downloaded chef11 data into a new directory and call transform on it if you wish too.
Ok, I will try. Actually I have this configuration:
Also does chef12-upgrade-data-transform supports `skip if exists’ mode, like when client , node is already there do not re-create it … I mean what if I run chef12-upgrade-data-transform more the once against the same chef12 server. I would then need that objects previously created would not be re-created again …
How this could be managed ? manually ?
Yes, it’s a manual process, I’ve the feeling we’re discussing an XY problem there.
What are you trying to achieve and why does a complete migration do not fit your need ?
I have a chef11 server with a lot of nodes, clients.
I need to replace chef11 by chef12 server.
I can’t do it in a one shot.
I need to to do it node (client) by node basis.
Say I choose node A and decide to migrate it to chef12. I do it ( in a one shot migration ). That’s ok. I go to node A server and change chef client config to use new chef12 server. How every the “side” effect of such a migration is now chef12 server has entries for all other nodes, clients comes from chef11 server.
A week later I choose another node B and decide to migrate it to chef12, when doing migration for node B, what is going to be with node A? Questions here - won’t migration failed because node, client already exists ( node A )? Also I don’t need to update node A data at all, when doing migration for node B
Does it make a sense ?
Once you migrated all nodes and their respective keys, the node by node migration is just updating their client.rb file to point to the new server (node attributes could become out of sync between the two servers, but unless you abuse normal levels attributes it should not be a problem).
The only point of attention when you have nodes on both server are:
The upload command won’t fail, but it will probably overwrite the existing nodes object, but as above, the only warning is about normal attributes which are part of the node object.
If you want to ensure a node by node update without loosing updated node attributes since the migration you can:
knife node show <the node> -F json > <the node>.json agains the chef11 serverknife node from file <the node>.json against the chef12 serverThis will update the node object without touching the client key nor acl, so it should be ok.
Honestly I migrated my whole server and it’s 300 nodes in one shot, I had no real problem, just small outside cases on cookbooks with weird method in their metada.rb.
I started on an opensource chef-server 11.0.8, made a migration to 11.1.6 and then migrated to 12.2, my only drawbacks was on the cookbooks themselve, chef clients above 11.10 are ok with a chef-server 12.
OK, I will probably try your way, thanks. Will let you know in any issues. Thanks again
HI!
[root@******* ~]# chef-server-ctl chef12-upgrade-download -s https://:443 -u *******-k ******** -d /tmp/chef11
Proceeding with options #<OpenStruct skip_setup=false, skip_download=false, skip_cleanup=false, chef11_server_url="https://:443", chef11_admin_client_name="", chef11_admin_client_key="*.pem", chef11_data_dir="/tmp/chef11">
Ensuring Chef 12 server is stopped
ok: down: bookshelf: 73s, normally up
ok: down: nginx: 73s, normally up
ok: down: oc_bifrost: 73s, normally up
ok: down: oc_id: 72s, normally up
ok: down: opscode-chef-mover: 72s, normally up
ok: down: opscode-erchef: 71s, normally up
ok: down: opscode-expander: 71s, normally up
ok: down: opscode-expander-reindexer: 70s, normally up
ok: down: opscode-solr4: 69s, normally up
ok: down: postgresql: 69s, normally up
ok: down: rabbitmq: 68s, normally up
ok: down: redis_lb: 68s, normally up
sh: /opt/chef-server/bin/wait-for-rabbit.tmp: No such file or directory
Failed to write fix to wait-for-rabbit script
Aww, sorry, forgot the command line flag -D for download only so it won’t try to ensure the chef11 server is running (case when you’re locally upgrading)
It seems this helped. Thank you.
[root@ip-10-147-137-46 ~]# chef-server-ctl chef12-upgrade-download -s https://*******:443 -u -k .pem -d /tmp/chef11 -D
Proceeding with options #<OpenStruct skip_setup=true, skip_download=false, skip_cleanup=true, chef11_server_url="https://:443", chef11_admin_client_name="", chef11_admin_client_key="******", chef11_data_dir="/tmp/chef11">
Timeout waiting for Chef 11 server to start. Received exception Connection refused - connect(2) for “localhost” port 8000
Seems strange it can’t get it, which version is your Chef11 server ? (is it an enterprise chef server 11 ?)
chef-server.x86_64 11.0.12-1.el6
@Melezhik Your Chef server is one version too old for the tooling to work. It needs to be at least 11.1 or newer. I’d recommend upgrading to 11.1.7, the most current of the 11x servers, then try again. Sorry this wasn’t clearer earlier, but the supplied upgrade tooling only works on 11.1 or greater.
Guys, thanks much for support. I decided to go abit another way.
Thanks for the follow up @mmzyk. I was totally missing the version point.