How to Upload to Private Supermarket Chef

When Chef unveiled Supermarket in 2014 information technology was intended to be a public community cookbooks site where Chef Community Members could scan cookbooks from anywhere. We were honored that it was received so well!

However, we soon discovered a trouble. Some companies create very company-specific cookbooks that they cannot share publicly (often for license or security reasons), nonetheless they nonetheless want a way to share them internally.

Before Individual Supermarket there were 2 chief means of sharing internal cookbooks. The kickoff was by storing cookbooks on Github Enterprise, merely the problem with this approach is information technology is difficult to scan cookbooks without knowing exactly what to search for. The second way was to use Berkshelf with Github Enterprise. The problem with this approach is that resolving dependencies from diverse Github Enterprise repos is slow.

At the Chef Community Summit 2014 Chef Community members requested the ability to run their ain copy of Supermarket on their own infrastructure. We hadn't idea of this need…just were certainly willing to try information technology! We first congenital a stand alone Supermarket package using Chef Passenger vehicle Packager.

Benefits of a Private Supermarket

A private supermarket provides an hands searchable cookbook repository (with friendly GUI and command line interfaces) that can run on a company's internal network. When Berkshelf is connected to a Private Supermarket's API, Berkshelf runs speed upwards. Customers take also told us it is easier to formalize the process of releasing cookbooks. If a new cookbook version is not all the same on the Private Supermarket, information technology is considered nether development or review and has not notwithstanding been released. If it is on Private Supermarket, it is considered officially released and set to utilize.

Installing a Individual Supermarket

Requirements

To install a private Supermarket, you volition need:

A Running Chef Server (this is needed for using oc-id – Chef's Oauth Provider)
An Admin User account on the Chef Server
A Cardinal for the User Account on Chef Server
A fresh VM with at to the lowest degree 1 GB memory (we use an Ubuntu instance on AWS)

What is oc-id?

oc_id is Chef'southward Authentication/Authorization service. It allows users to utilise their credentials for a Chef Server to access other applications.

For an intro to oc_id, please see oc-id on Chef Server: An Introduction.

Get-go, a user visits an application and clicks "Sign In"

And then they are redirected to the Chef Server where they sign in.

And then they are redirected back to the application, now signed in!

Preparing Your Chef Server

Starting time, let'due south ready the Chef Server to permit your Private Supermarket to use it for authentication/authorization. Go ahead and ssh in:

$ (your workstation) ssh user@your-chef-server

Then open upwardly this file:

$ (your chef server) sudo vim /etc/opscode/chef-server.rb

Add this content, then relieve and shut the file:

                      ocid['applications'] = {     'supermarket' => {        'redirecturi' => 'https://supermarket-hostname/auth/chef_oauth2/callback'      }   }

Now reconfigure your Chef Server:

$ (your chef server) sudo chef-server-ctl reconfigure

Once that completes, take a await at this file:

$ (your chef server) sudo true cat /etc/opscode/oc-id-applications/supermarket.json

You should see output that looks like this:

                      {    "proper name": "supermarket",    "uid": "123...989",    "surreptitious": "1234...897",    "redirecturi": "https://supermarket-hostname/auth/chefoauth2/callback"  }

Have special notation of the "uid" and "secret" values – you will demand these later. (Note: You can always get them once more by looking at this file on your Chef Server, they don't go away after you view them the kickoff time.)

Preparing Your Supermarket

The all-time cookbook to use when setting up a Supermarket is the supermarket-omnibus-cookbook.

It's good practice to use a wrapper cookbook around the supermarket-omnibus-cookbook, we will need to supply certain attributes through the wrapper cookbook.

Let's get through the layers of the wrapper cookbook, cookbook, and internal cookbook inside the Passenger vehicle package.

Wrapper Cookbook

First there is the wrapper cookbook where nosotros define node[supermarket_omnibus] attributes:

supermarket-omnibus-cookbook

Then there is the supermarket-bus-cookbook, which is what our wrapper cookbook wraps around. This cookbook will:

Install the Supermarket Bus yum or deb packages
Writes the node[supermarket_omnibus] attributes to /etc/supermarket/supermarket.json

Supermarket Omnibus Bundle Internal Cookbook

Finally, we take the Supermarket Omnibus yum or deb bundle. This parcel has an internal chef cookbook which installs and configures the parcel using the attributes defined in /etc/supermarket/supermarket.json. When the package runs this internal cookbook information technology is very similar to running chef solo on a server.

Creating the Wrapper Cookbook

Then permit's go ahead and create the wrapper cookbook.

On your workstation, generate a new cookbook.

$ (your workstation) chef generate cookbook my-supermarket-wrapper

Then change directories into that cookbook:

$ (your workstation) cd my-supermarket-wrapper

Now get alee and open the metadata file of the cookbook:

$ (your workstation) vim metadata.rb

And add this line, then save and close the file. This defines our wrapper cookbook's dependency on the supermarket-omnibus-cookbook:

                      depends 'supermarket-charabanc-cookbook'

At present open up the default recipe within the cookbook:

$ (your workstation) vim recipes/default.rb

And add this content, which volition execute the default recipe of the supermarket-omnibus-cookbook

                      include_recipe 'supermarket-charabanc-cookbook'

Next we need to ascertain the attributes for the Supermarket installation and how it connects to the Chef Server. Ane solution is to difficult code attributes in the wrapper cookbook's default recipe, but a meliorate practise is to place the attributes in a information purse (or encrypted information bag or vault), then reference them in them recipe.

At a minimum, we must define the chefserverurl, chefoauth2app, chefoauth2secret attributes.

Then permit's open up up the default recipe again:

$ (your workstation) vim recipes/default.rb

And let'south pretend we have a information bag called 'apps', with an detail in it called 'supermarket':

                      # calling the information bag   app = databagitem('apps', 'supermarket')

Then let's add in attributes we desire to utilise from the databag:

                      # calling the information bag   app = databagitem('apps', 'supermarket')   node.set['supermarketomnibus']['chefserverurl'] = app['chefserverurl']   node.fix['supermarketomnibus']['chefoauth2appid'] = app['appid']   node.set['supermarketomnibus']['chefoauth2_secret'] = app['secret']

Go ahead and save and close the file.

Now, permit'due south install the dependent cookbooks in supermarket-charabanc-cookbook.

$ (your workstation) berks install

Now we need to upload all dependent cookbooks to the Chef Server (NOTE: There is more than than one mode to practice this, employ whatever works best for you and your team's workflow).

$ (your workstation) cd ~/.berkshelf/cookbooks

$ (your workstation) knife cookbook upload -a

Then upload the wrapper cookbook (again, there is more than one way of doing this, this is one way that works):

$ (your workstation) cd path/to/wrapper/cookbook/

$ (your workstation) knife cookbook upload -a

Now let'south bootstrap our Supermarket node with the Chef Server. If we had an ubuntu node in AWS, we would bootstrap it like this:

$ (your workstation) pocketknife bootstrap ip_address -Due north supermarket-node -x ubuntu --sudo

–N flag defines the name of the node (in this case, supermarket-node). –x flag defines the username to use (the default username for Ubuntu instances on AWS is Ubuntu). --sudo runs the bootstrap command as sudo on the node.

Once bootstrapping is complete, we edit the new supermarket node

$ (your workstation) pocketknife node edit supermarket-node

And add the wrapper's default recipe to the supermarket-node's run list and then save and quit the file.

                      "runlist": [     "recipe[mysupermarket_wrapper::default]"   ]

Now nosotros ssh into the Supermarket node

$ (your workstation) ssh ubuntu@your-supermarket-node-public-dns

And one time nosotros're in the node, nosotros run chef-client. This will install and configure Supermarket.

$ (your supermarket node) sudo chef-client

Using Private Supermarket

Connecting to Supermarket

When nosotros're ready to use our newly spun up Private Supermarket, nosotros need to open up upwards /etc/hosts on our local workstation (the workstation nosotros'll be uploading cookbooks from to Supermarket) and add together an entry for our Supermarket instance:

                      # Supermarket ip address Supermarket hostname   00.00.000.000 supermarket-hostname

After saving and closing the file, nosotros visit the Supermarket hostname in the browser (Supermarket past default uses a cocky-signed certificate, if we receive a alarm we need to have the SSL certificate) and so click the "Create Account" link. If non already logged into the Chef Server, we will be prompted to exercise so. Then we need to qualify the supermarket app to use our Chef Server account and we're in!

Interacting with Cookbooks on Supermarket

The best current and Chef supported way to interact with a private Supermarket is with the knife-supermarket plugin. (NOTE: Other tools can be used and nosotros volition go over those a lilliputian later).

First, nosotros install knife-supermarket.

If Using the Chef DK

$ (your workstation) chef gem install knife-supermarket

If Not Using the Chef DK

$ (your workstation) precious stone install pocketknife-supermarket

Now we open our pocketknife.rb file:

$ (your workstation) vim .chef/knife.rb

And define the supermarket site for our Private Supermarket, then save and close the file:

                      pocketknife[:supermarket_site] = 'https://your-individual-supermarket'

Knife Supermarket commands are the same as knife cookbook site commands, only with the power to connect with an Private Supermarket rather than merely the Public Supermarket. Please consult the docs for information on all commands that can be used with Knife Supermarket.

Let'south accept the time to get over how to share a cookbook to a private Supermarket. Using knife supermarket, we would run this command:

$ (your-workstation) knife supermarket share 'my_cookbook'

When we get-go run this command, we might come across an SSL error:

          $ (your-workstation) knife supermarket share 'my_cookbook'  Making tarball my_cookbook.tgz  Error: Fault uploading cookbook mycookbook to the Opscode Cookbook Site: SSLconnect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed. Increase log verbosity (-VV) for more information.

This is because Chef 12 enforces SSL past default when sharing cookbooks (Chef eleven did Not do this). Private Supermarkets, past default, apply self-signed certificates. Fortunately nosotros can fix this error through fetching and checking the private Supermarket'southward ssl certificate.

$ (your-workstation) knife ssl fetch https://your-individual-supermarket

Followed by:

$ (your-workstation) knife ssl check https://your-individual-supermarket

At present let's attempt sharing the cookbook again and we should see a success message:

          $ (your-workstation) knife supermarket share 'my_cookbook'  Generating metadata for my_cookbook from (...)  Making tarball my_cookbook.tgz  Upload complete!

Managing Private Supermarket

supermarket-ctl

We tin manage services and other settings of our instance with supermarket-ctl commands.

For example, running supermarket-ctl reconfigure will do an internal Chef run based on the cookbook within the omnibus packet.

supermarket-ctl brand-admin username will make a supermarket user and admin user

supermarket-ctl restart will stop services if they are running, and then start them again.

Hither's what we would run across when using supermarket-ctl restart on a supermarket server:

          $ (your supermarket node) sudo supermarket-ctl restart  ok: run: nginx: (pid ####) 1s  ok: run: postgresql: (pid ####) 0s  ok: run: rails: (pid ####) 1s  ok: run: redis: (pid ####) 0s  ok: run: sidekiq: (pid ####) 0s

Monitoring

By default every installation of Supermarket includes a Monitoring URL at https://yourindividualsupermarket/status.

Other Tools With Private Supermarket

Stove

Stove is an alternate utility for releasing and managing Chef Cookbooks.

Berkshelf

Berkshelf can include multiple Supermarkets for dependency resolution. Cookbook dependency resolution is performed from the summit downward – the first source defined in the Berksfile will be searched for the cookbook before the second source.

This Berksfile would beginning expect for the cookbook on the individual Supermarket and, if it did not find a cookbook with the specified name, would wait on the public Supermarket.

                      source 'https://yourprivatesupermarketurl'   source 'https://supermarket.chef.io'

Further Customizing a Private Supermarket

Configure an external database

Supermarket installations can use an external database (we use Amazon RDS for the Public Supermarket). To configure this, we would need to configure the following attributes in the default recipe of the wrapper cookbook.

          omnibus']['postgresql']['enable'] = false   node.gear up['supermarketomnibus']['database']['user']     = 'supermarket'   node.set['supermarketomnibus']['database']['proper noun']     = 'supermarket'   node.prepare['supermarketomnibus']['database']['host']     = 'yourcompany...rds.amazon.com'   node.fix['supermarketomnibus']['database']['port']     = '5432'   node.set['supermarketomnibus']['database']['puddle']     = '25'   node.set['supermarketomnibus']['database']['countersign'] = 'topsecretneverguessit'

Configure an External Cache

Supermarket installations can also use an external enshroud store. Public Supermarket uses redis on Amazon Elasticache. To configure this, we would need to configure the following attributes in the default recipe of the wrapper cookbook.

                      node.set['supermarketomnibus']['redis']['enable'] = false   node.set['supermarketomnibus']['redis_url'] = 'redis://your-redis-case:6379'

Configure Cookbook Storage

Cookbook artifacts (the tar.gz artifacts which are uploaded to Supermarket when sharing a cookbook) can be stored either on the local filesystem of the Supermarket node (this is the default) or in an Amazon S3 bucket.

If we were using an Amazon S3 bucket, we would configure the post-obit attributes in the default recipe of the wrapper cookbook.

                      node.set['supermarketomnibus']['s3accesskeyid'] = false   node.fix['supermarketomnibus']['s3bucket'] = 'supermarket'   node.set['supermarketomnibus']['s3accesskeyid'] = 'yoursecretaccesskey'

Additional Configuration Options

For boosted configuration options for a individual Supermarket, delight consult this list of default attributes included with Omnibus Supermarket. And of these attributes can be configured.

Decision

Supermarket — both the public and individual versions — was created for the Chef Community. If yous have problems, suggestions, or feature requests delight consider opening an consequence in the Supermarket Github Repo.

Note: The content for this blog post was originally developed for a webinar in collaboration with Kurt Fitzpatrick.

dicksonnotenjoyard.blogspot.com

Source: https://dzone.com/articles/a-supermarket-of-your-own-running-a-private-superm