How to load users into new chef 12 server from chef 11 server

Chef Infra (archive)
1

Hello. I’m building a chef 12 standalone server, and trying to knife upload all
of our data into it from our chef 11 server.

How do I load users from the chef 11 server to chef 12? I’m trying, but failing
as seen below. This technique has worked for us when uploading users to a chef 11 server.

And, for these uploaded users, to what Org are they associated? Since the ORG_NAME endpoint is in my knife.rb below, I assume it uses that Org? (And how do I make some of theses users admin users? so many chef 12 questions…)

thanks
kallen

Here are the details: … sigh, markup, html, sigh.

I’m following instrux here: https://docs.chef.io/release/server_12-2/install_server.html#standalone

version chef-server-core-12.2.0-1.el6.x86_64

Create the first user. Create the first org (and apparently associate that new user to that org)

Side question: is this first new user considered to have admin level privs for this Org?

# chef-server-ctl user-create devops Dev Ops ops@boop.com REDACTED --filename /root/devops.pem
# chef-server-ctl org-create org1 "Dev Ops Org1" --association_user devops --filename /root/org1-validator.pem

And now, try to upload a user:

# cat /root/knife.rb
chef_server_url "https://localhost/organizations/org1"
node_name "devops"
client_key "/etc/opscode/devops.pem"
#repo_mode "everything"
versioned_cookbooks true
chef_repo_path "/ebs/chef/restore/transfer"
cookbook_path "/ebs/chef/restore/transfer/cookbooks"
ssl_verify_mode :verify_none

# knife user list -c /root/knife.rb
devops

# cat kallen.json
{
  "name": "kallen",
  "admin": true,
  "public_key": "-----BEGIN RSA PUBLIC KEY-----\nMIIBCg..snip \n",
  "email": "kallen@boop.com",
  "password": "REDACTED"
}

# knife upload kallen.json -c /root/knife.rb -VV
INFO: Using configuration from /root/knife.rb
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/chef_fs/file_system/base_fs_object.rb:124:in `create_child': Chef::ChefFS::FileSystem::OperationNotAllowedError (Chef::ChefFS::FileSystem::OperationNotAllowedError)
        from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/chef_fs/file_system.rb:417:in `get_or_create_parent'
        from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/chef_fs/file_system.rb:144:in `block in copy_to'
        from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/chef_fs/parallelizer/parallel_enumerable.rb:267:in `call'
        from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/chef_fs/parallelizer/parallel_enumerable.rb:267:in `process_input'
        from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/chef_fs/parallelizer/parallel_enumerable.rb:257:in `process_one'
        from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/chef_fs/parallelizer.rb:93:in `call'
        from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/chef_fs/parallelizer.rb:93:in `worker_loop'
2

Hi Kellan -

I recommend you look at my reply to another thread here: Migrating data from exitsted chef11 servers to new chef12 server (I think this link will work).

You can use the upgrade tooling packaged with the Chef 12 server to do what you want, you’ll just need to add a manual step of copying over the data from the old server to the new one.

As far as how that will setup an org, you’ll set a default org that all the migrated data will be associated with during that process.

If anything isn’t clear, ask and I’ll try to provide clarification. Do note that for the tooling to work your Chef 11 server has to be at least version 11.1 or greater. If it isn’t, upgrade to the latest 11 version and then proceed. (I’m assuming you’re running open source Chef 11 server and not Enterprise Chef 11).

Thanks,

Mark Mzyk

3

Hi there,

The thing is, we don’t want to upgrade our chef 11 server in place. We want to
build a fresh chef 12 server and load data into it from backups. Given this
dir structure, this all works, except for uploading the users.

[transfer]# ls -l
total 132
drwxr-xr-x 2 root root 40960 Oct 21 18:20 clients
drwxr-xr-x 592 root root 32768 Oct 21 18:30 cookbooks
drwxr-xr-x 21 root root 4096 Oct 21 18:20 data_bags
drwxr-xr-x 2 root root 4096 Oct 21 18:20 environments
drwxr-xr-x 2 root root 40960 Oct 21 18:21 nodes
drwxr-xr-x 2 root root 4096 Oct 21 18:20 roles
drwxr-xr-x 2 root root 4096 Oct 21 18:20 users

[transfer]# knife upload / -c /root/knife.rb

4

Let me apologize for misspelling your name in my other post. Sorry about that Kallen.

Yes, what I linked to talks about how to do what you want. The upgrade tooling will allow you to do this. You install the Chef 12 package on the Chef 11 server, but never run reconfigure. This will give you access to the upgrade commands, but without changing your Chef 11 server (Chef 11 and Chef 12 servers install in different directories - they can live side by side, although both can’t be running at the same time. In fact, the automated upgrade when run against a Chef 11 server is a full download from the Chef 11 server, transform the data, and upload to the Chef 12 server all on the same box). Installing the Chef 12 package on a Chef 11 server will change the default path to chef-server-ctl to that of Chef 12, but you can use the direct path of /opt/chef-server/bin/chef-server-ctl to continue to access the Chef 11 server’s chef-server-ctl (Chef 12 installs to /opt/opscode/).

With the tooling in place, you have access to the upgrade subcommands: https://docs.chef.io/upgrade_server_open_source_notes.html#manual-upgrades

These are the three steps that make up an upgrade. So you can run the download step, then the transform step on your Chef 11 server. This will save your data by default into a folder under /tmp, but you can specify the location via a command line option. Transforming the data will move the data into the format expected by the Chef 12 server and will again by default save it to a folder under /tmp. (As an aside, this data is mostly json laid out in a particular folder structure - so you can go in and inspect it and even modify it if needed). The output will tell you the name of this folder if you don’t change the default. Then you copy this transformed data over to your new Chef 12 server and the on the Chef 12 server run the upload step to complete the process. The three commands I’ve described here were put in place exactly for a scenario just like this. If you run chef-server-ctl upgrade --help you can see all the options associated with the commands. We attempted to make this process very flexible to enable various upgrade scenarios. The one thing we didn’t get implemented is running the process on a new Chef 12 server box and pulling from a remote Chef 11 server box. I’d wish we’d gotten to that, but we didn’t, so for now it remains a manual step to copy over the data.

If you don’t want to do this and want to work with what you already have, the entire code that runs the upgrade process is here: https://github.com/chef/chef-server/blob/master/omnibus/files/private-chef-ctl-commands/open_source_chef12_upgrade.rb so you can look at it to see how to extract, transform, and upload just the users from your system.

In particular, the user object changes a fair amount between open source Chef 11 and the Chef 12 server, which is why doing a straight download and upload doesn’t work. See https://github.com/chef/chef-server/blob/master/omnibus/files/private-chef-ctl-commands/open_source_chef12_upgrade.rb#L431 for what the upgrade process does to get users into a format that works for Chef 12. Note that that code snippet isn’t enough, as the upgrade process also pulls credentials from the Chef 11 db to ensure that users can continue to function as needed in Chef 12: https://github.com/chef/chef-server/blob/master/omnibus/files/private-chef-ctl-commands/open_source_chef12_upgrade.rb#L387

Hopefully this answers your question and tells you how to do what you want. If you have any other questions I’ll do my best to answer them.

Mark Mzyk
Chef Software Engineer

5

Just as side note: as long as your chef client is over 11.8 (IIRC, but prefer a 11.18 if possible, personnaly Iit went well with a 11.16.4) there’s no problem migrating in place to a chef12 server from opsource chef11 server.

The migrations process is correct enough so you can rollback at any time to the chef server 11 if anything goes bad.

6

Thanks for these details. I'll dig into this today!

On Thu, 22 Oct 2015, mmzyk wrote:

Let me apologize for misspelling your name in my other post. Sorry about that Kallen.

no worries :>