Placing large binary files in a cookbook is discouraged so we've been placing them on an on-prem Apache server, but nodes behind a firewall are not allowed access to this web server causing issues. Since all nodes (including those behind the firewall) do have access to our Chef-Infra server, is there any way to place these large files on the Chef-Infra server for easier downloading? So either in some Chef construct or by installing/setting up Apache on the Chef-Infra server (would need to be the same 443 port that Chef-Infra is already using)?
I know this has been asked before, but I'm not finding any answers so any pointers to answers would be appreciated.
I would highly discourage it, but you an always install and configure a website to be hosted on another port that isn't being used and use that for file downloads.
Hi Tait,
I'm in the same camp as Stromweld on the chef server. It's better to build another server along side your chef server that will serve the files. However if you really can't do that you can setup an apache server on the chef server that serves files over a different port like 10443 since chef is using 443 already. Then you set your download url to have ":10443" appended to the hostname. Once again your chef server should only be your chef server. File server should run on another server. FYI it's also a good idea to make your file server internal only and not publically available. Not saying you did that but it being on the other side of a firewall suggests it is.
So no other solution? Why is this not recommended? Having a different port defeats the purpose since that too would require an additional firewall rule to allow the new port and it would be helpful to be able to re-use the default 443 port without having to submit an additional firewall port request.
I have found that on the Chef-Infra server I can place files in /var/opt/opscode/nginx/html/images and I'm then able to download the files on 443, but I'd prefer an officially supported mechanism. So is the answer that Chef does not officially support this and we are left to implement our own solution requiring our own fileserver?
Note: We've internal servers and external servers and the external servers have to go through a firewall to get to the Chef-Infra server and though the firewall rules should be allowing both our Chef-Infra server and our fileserver server, more and more often the rules only implement access to the Chef-Infra server. So when a new server is bootstrapped it fails and then I'm called in to help (as the last resort) and by the time the request comes to me it has become an 'emergency' to get the new server built immediately. Yes, Ideally the firewall rule allowing access to the fileserver would be opened, but by the time I get involved it needs to be fixed asap so I often am forced to manually download the needed files to the new node to manually install just to chef-client can run. Part of the problem is justifying to the network team the need for firewall rules to the fileserver just once doing the initial bootstrapping and not really needed after that.
So my hope was there was some officially supported mechanism in Chef that could be used to provide access to large files. Clearly looks like this is NOT supported forcing me to tell people that the needed firewall rules MUST be opened for the bootstrapping to work.
Hi Tait,
Using Chef server as a File server is not officially supported because Chef is a "Configuration Management Server". Using the service over 443 that chef uses for a file server is not recommended. Setting up a file server on an different server is just "Best Practices" in our field. Sorry your network is a pain, to me this is just business as usual. In the >25 years I've been doing this I've found there is more than one way to make it work. It's up to you to go with what works best for your network. Good luck.