# Automatically Discover GCP Compute Instances

This guide shows you how to set up automatic server discovery for Google Compute Engine virtual machines.

## How it works

The Teleport Discovery Service can connect to GCP and automatically discover and enroll GCP Compute Engine instances matching configured labels. It will then execute a script on these discovered instances that will install Teleport, start it and join the cluster.

## Prerequisites

- A running Teleport cluster. If you want to get started with Teleport, [sign up](https://goteleport.com/signup) for a free trial or [set up a demo environment](https://goteleport.com/docs/get-started/deploy-community.md).

- The `tctl` and `tsh` clients.

  Installing `tctl` and `tsh` clients

  1. Determine the version of your Teleport cluster. The `tctl` and `tsh` clients must be at most one major version behind your Teleport cluster version. Send a GET request to the Proxy Service at `/v1/webapi/find` and use a JSON query tool to obtain your cluster version. Replace teleport.example.com:443 with the web address of your Teleport Proxy Service:

     ```
     $ TELEPORT_DOMAIN=teleport.example.com:443
     $ TELEPORT_VERSION="$(curl -s https://$TELEPORT_DOMAIN/v1/webapi/find | jq -r '.server_version')"
     ```

  2. Follow the instructions for your platform to install `tctl` and `tsh` clients:

     **Mac**

     Download the signed macOS .pkg installer for Teleport, which includes the `tctl` and `tsh` clients:

     ```
     $ curl -O https://cdn.teleport.dev/teleport-${TELEPORT_VERSION?}.pkg
     ```

     In Finder double-click the `pkg` file to begin installation.

     ---

     DANGER

     Using Homebrew to install Teleport is not supported. The Teleport package in Homebrew is not maintained by Teleport and we can't guarantee its reliability or security.

     ---

     **Windows - Powershell**

     ```
     $ curl.exe -O https://cdn.teleport.dev/teleport-v${TELEPORT_VERSION?}-windows-amd64-bin.zip
     Unzip the archive and move the `tctl` and `tsh` clients to your %PATH%
     NOTE: Do not place the `tctl` and `tsh` clients in the System32 directory, as this can cause issues when using WinSCP.
     Use %SystemRoot% (C:\Windows) or %USERPROFILE% (C:\Users\<username>) instead.
     ```

     **Linux**

     All of the Teleport binaries in Linux installations include the `tctl` and `tsh` clients. For more options (including RPM/DEB packages and downloads for i386/ARM/ARM64) see our [installation page](https://goteleport.com/docs/installation.md).

     ```
     $ curl -O https://cdn.teleport.dev/teleport-v${TELEPORT_VERSION?}-linux-amd64-bin.tar.gz
     $ tar -xzf teleport-v${TELEPORT_VERSION?}-linux-amd64-bin.tar.gz
     $ cd teleport
     $ sudo ./install
     Teleport binaries have been copied to /usr/local/bin
     ```

* A GCP compute instance to run the Discovery Service on.
* GCP compute instances to join the Teleport cluster, running Ubuntu/Debian/RHEL if making use of the default Teleport install script. (For other Linux distributions, you can install Teleport manually.)
* To check that you can connect to your Teleport cluster, sign in with `tsh login`, then verify that you can run `tctl` commands using your current credentials. For example, run the following command, assigning teleport.example.com to the domain name of the Teleport Proxy Service in your cluster and email\@example.com to your Teleport username:
  ```
  $ tsh login --proxy=teleport.example.com --user=email@example.com
  $ tctl status
  Cluster  teleport.example.com
  Version  19.0.0-dev
  CA pin   sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678
  ```
  If you can connect to the cluster and run the `tctl status` command, you can use your current credentials to run subsequent `tctl` commands from your workstation. If you host your own Teleport cluster, you can also run `tctl` commands on the computer that hosts the Teleport Auth Service for full permissions.

## Step 1/5. Create a GCP invite token

When discovering GCP compute instances, Teleport makes use of GCP invite tokens for authenticating joining SSH Service instances.

Create a file called `token.yaml`:

```
# token.yaml
kind: token
version: v2
metadata:
  # the token name is not a secret because instances must prove that they are
  # running in your GCP project to use this token
  name: gcp-discovery-token
spec:
  # use the minimal set of roles required (e.g. Node, Proxy, App, Kube, DB, WindowsDesktop)
  roles: [Node]

  # set the join method allowed for this token
  join_method: gcp

  gcp:
    allow:
      # The GCP project ID(s) that VMs can join from.
      - project_ids: []
        # (Optional) The locations that VMs can join from. Note: both regions and
        # zones are accepted.
        locations: []
        # (Optional) The email addresses of service accounts that VMs can join
        # with.
        service_accounts: []

```

Add your instance's project ID(s) to the `project_ids` field. Add the token to the Teleport cluster with:

```
$ tctl create -f token.yaml
```

## Step 2/5. Configure IAM permissions for Teleport

Create a service account that will give Teleport IAM permissions needed to discover instances.

**Console**

Go to [IAM > Roles](https://console.cloud.google.com/iam-admin/roles) in the GCP console and click *+ Create Role*. Pick a name for the role (e.g. `teleport-discovery`) and give it the following permissions:

- `compute.instances.get`
- `compute.instances.getGuestAttributes`
- `compute.instances.list`
- `compute.instances.setMetadata`
- `iam.serviceAccounts.actAs`
- `iam.serviceAccounts.get`
- `iam.serviceAccounts.list`

![Custom role](/docs/assets/images/custom-role@2x-413c5b97560d94af8657abec8a9247a7.png)

Click *Create*.

Go to [IAM > Service accounts](https://console.cloud.google.com/iam-admin/serviceaccounts) and click *+ Create Service Account*. Pick a name for the service account (e.g. `teleport-discovery`) and copy its email address to your clipboard. Click *Create and Continue*.

![Service account](/docs/assets/images/create-service-account@2x-ca422c7e72f278e4487b039366b6890f.png)

Go to [IAM](https://console.cloud.google.com/iam-admin/iam) and click *Grant Access*. Paste the service account's email into the *New principals* field and select your custom role. Click *Save*.

![Role assignment](/docs/assets/images/assign-role-to-service-account@2x-a1f456abd95d6d768fbef1ee86cf346d.png)

If the Discovery Service will run in a GCP compute instance, edit the instance and assign the service account to the instance and set its access scopes to include Read Write access to the Compute API.

![Service account assignment](/docs/assets/images/access-scopes@2x-91976aebe7ce4e4d5aba31b75b5e1b5a.png)

**gcloud**

Copy the following and paste it into a file called `teleport-discovery-role.yaml`:

```
# teleport-discovery-role.yaml
title: "teleport-discovery"
description: "A role to enable Teleport to discover GCP compute instances"
stage: "ALPHA"
includedPermissions:
- compute.instances.get
- compute.instances.getGuestAttributes
- compute.instances.list
- compute.instances.setMetadata
- iam.serviceAccounts.actAs
- iam.serviceAccounts.get
- iam.serviceAccounts.list

```

Then run the following command to create the role:

```
$ gcloud iam roles create teleport_discovery \
--project=project_id \
--file=teleport-discovery-role.yaml
```

Run the following command to create the service account:

```
$ gcloud iam service-accounts create teleport-discovery \
--description="A service account to enable Teleport to discover GCP compute instances" \
--display-name="teleport-discovery"
```

Run the following command to add the new role to the new service account:

```
$ gcloud projects add-iam-policy-binding project_id \
--member="serviceAccount:teleport-discovery@project_id.iam.gserviceaccount.com" \
--role="projects/project_id/roles/teleport_discovery"
```

If the Discovery Service will run in a GCP compute instance, run the following command to add the service account to the instance, replacing discovery\_service\_vm\_name with the name of the Discovery Service VM:

```
$ gcloud compute instances set-service-account discovery_service_vm_name \
--service-account=teleport-discovery@project_id.iam.gserviceaccount.com \
--scopes=default,compute-rw
```

## Step 3/5. Configure instances to be discovered

Ensure that each instance to be discovered has a service account assigned to it. No permissions are required on the service account. To check if an instance has a service account, run the following command and confirm that there is at least one entry under `serviceAccounts`:

```
$ gcloud compute instances describe --format="yaml(name,serviceAccounts)" instance_name
```

### Enable guest attributes on instances

Guest attributes (a subset of custom metadata used for infrequently updated data) must be enabled on instances to be discovered so Teleport can access their SSH host keys. Enable guest attributes by setting `enable-guest-attributes` to `TRUE` in the instance's metadata.

```
$ gcloud compute instances add-metadata instance_name \
--metadata=enable-guest-attributes=True
```

If guest attributes are enabled during instance creation, the guest attributes will automatically be populated with the instance's host keys. If guest attributes were enabled after the instance was created, you can manually add the host keys to the guest attributes below:

**Startup script**

Create a file named `add-host-keys.sh` and copy the following into it:

```
#!/usr/bin/env bash
for file in /etc/ssh/ssh_host_*_key.pub; do
  read -r KEY_TYPE KEY _ <"$file"
  curl -X PUT --data "$KEY" "http://metadata.google.internal/computeMetadata/v1/instance/guest-attributes/hostkeys/$KEY_TYPE" -H "Metadata-Flavor: Google"
done

```

Run the following command to add the host keys as part of a startup script:

```
$ gcloud compute instances add-metadata instance_name \
--metadata-from-file=startup-script="add-host-keys.sh"
```

**SSH**

Run the following command to add the host keys over SSH:

```
$ gcloud compute ssh instance_name \
--command='for file in /etc/ssh/ssh_host_*_key.pub; do KEY_TYPE=$(awk '\''{print $1}'\'' $file); KEY=$(awk '\''{print $2}'\''  $file); curl -X PUT --data "$KEY" "http://metadata.google.internal/computeMetadata/v1/instance/guest-attributes/hostkeys/$KEY_TYPE" -H "Metadata-Flavor: Google"; done'
```

## Step 4/5. Install the Teleport Discovery Service

---

TIP

If you plan on running the Discovery Service on a host that is already running another Teleport service (Auth or Proxy, for example), you can skip this step.

---

Install Teleport on the virtual machine that will run the Discovery Service.

To install a Teleport Agent on your Linux server:

The recommended installation method is the cluster install script. It will select the correct version, edition, and installation mode for your cluster.

1. Assign teleport.example.com:443 to your Teleport cluster hostname and port, but not the scheme (https\://).

2. Run your cluster's install script:

   ```
   $ curl "https://teleport.example.com:443/scripts/install.sh" | sudo bash
   ```

## Step 5/5. Configure Teleport to discover GCP compute instances

If you are running the Discovery Service on its own host, the service requires a valid invite token to connect to the cluster. Generate one by running the following command against your Teleport Auth Service:

```
$ tctl tokens add --type=discovery
```

Save the generated token in `/tmp/token` on the virtual machine that will run the Discovery Service.

---

WARNING

Discovery Service exposes a configuration parameter - `discovery_service.discovery_group` - that allows you to group discovered resources into different sets. This parameter is used to prevent Discovery Agents watching different sets of cloud resources from colliding against each other and deleting resources created by another services.

When running multiple Discovery Services, you must ensure that each service is configured with the same `discovery_group` value if they are watching the same cloud resources or a different value if they are watching different cloud resources.

It is possible to run a mix of configurations in the same Teleport cluster meaning that some Discovery Services can be configured to watch the same cloud resources while others watch different resources. As an example, a 4-agent high availability configuration analyzing data from two different cloud accounts would run with the following configuration.

- 2 Discovery Services configured with `discovery_group: "prod"` polling data from Production account.
- 2 Discovery Services configured with `discovery_group: "staging"` polling data from Staging account.

---

Assign teleport.example.com:443 to the host and port of the Teleport Proxy Service in your cluster, and gcp-prod to a name that identifies a group of resources that you will enroll:

```
# teleport.yaml
version: v3
teleport:
  join_params:
    token_name: "/tmp/token"
    method: token
  proxy_server: "teleport.example.com:443"
auth_service:
  enabled: false
proxy_service:
  enabled: false
ssh_service:
  enabled: false
discovery_service:
  enabled: true
  discovery_group: gcp-prod

```

Create a matcher for the resources you want to enroll.

---

TIP

Dynamic configuration uses Discovery Configs which can be managed using Terraform. See the [Terraform `discovery_config` reference](https://goteleport.com/docs/reference/infrastructure-as-code/terraform-provider/resources/discovery_config.md) for more information.

Static configuration while simpler at first, has less flexibility because enrollment changes require edits to `teleport.yaml` and the restart of the Discovery Service.

---

**Dynamic configuration (recommended)**

Create a Discovery Config resource, that has the same discovery group you configured earlier, to enable GCP VM discovery.

Create a file named `discovery-gcp-prod.yaml` with the following content:

```
kind: discovery_config
version: v1
metadata:
  name: example-discovery-config
spec:
  discovery_group: gcp-prod
  gcp:
    - types: ["gce"]
      # The IDs of GCP projects that VMs can join from.
      project_ids: []
      # (Optional) The locations that VMs can join from. Note: both regions and
      # zones are accepted.
      locations: []
      # (Optional) The email addresses of service accounts that VMs can join
      # with.
      service_accounts: []
      # (Optional) Labels that joining VMs must have.
      labels:
        "env": "prod" # Match virtual machines where label:env=prod

```

Adjust the keys under `spec.gcp` to match your GCP environment, specifically the project IDs, locations, service accounts and labels you want to associate with the Discovery Service.

Create the Discovery Config by running the following command:

```
$ tctl create -f discovery-gcp-prod.yaml
```

Matching instances will be added to the Teleport cluster automatically.

You can update the Discovery Config at any time, and the service will automatically re-apply the changes.

**Static configuration**

In order to enable GCP VM discovery the `discovery_service.gcp` section of `teleport.yaml` must include at least one entry:

```
# teleport.yaml
# ...
discovery_service:
  enabled: true
  discovery_group: gcp-prod
  gcp:
    - types: ["gce"]
      # The IDs of GCP projects that VMs can join from.
      project_ids: []
      # (Optional) The locations that VMs can join from. Note: both regions and
      # zones are accepted.
      locations: []
      # (Optional) The email addresses of service accounts that VMs can join
      # with.
      service_accounts: []
      # (Optional) Labels that joining VMs must have.
      labels:
        "env": "prod" # Match virtual machines where label:env=prod

```

Adjust the keys under `discovery_service.gcp` to match your GCP environment, specifically the projects, locations, service accounts, and tags you want to associate with the Discovery Service.

### GCP credentials

The Teleport Discovery Service must have the credentials of the `teleport-discovery` GCP service account we created above in order to be able to log in.

The easiest way to ensure that is to run the Discovery Service on a GCP instance and assign the service account to that instance. Refer to [Set Up Application Default Credentials](https://cloud.google.com/docs/authentication/provide-credentials-adc) for details on alternate methods.

## Auto-discovery labels

Teleport applies a set of default labels to resources on AWS, Azure, and Google Cloud that join a cluster via auto-discovery. See the auto-discovery labels [reference](https://goteleport.com/docs/enroll-resources/auto-discovery/reference/labels.md)

## Advanced configuration

This section covers configuration options for discovering and enrolling servers.

### Install multiple Teleport agents on the same instance

When using blue-green deployments or other multiple clusters setups, you might want to access your instances from different clusters.

Teleport supports installing and running multiple agents on the same instance, using a suffixed installation which allows you to isolate each installation.

**Dynamic configuration (recommended)**

To configure the Discovery Service to use a suffixed installation, edit the Discovery Config and set the `spec.gcp.install.suffix` key:

```
kind: discovery_config
# ...
spec:
  gcp:
   - install:
       suffix: "blue-cluster"

```

**Static configuration**

To configure the Discovery Service to use a suffixed installation, specify the `install.suffix` key in your Discovery Service configuration:

```
# teleport.yaml
version: v3
# ...
discovery_service:
  enabled: true
  gcp:
   - install:
       suffix: "blue-cluster"

```

Requires [agent managed updates](https://goteleport.com/docs/upgrading/agent-managed-updates.md) to be enabled.

### Define the group for Managed Updates

If you are using Teleport [Agent managed updates](https://goteleport.com/docs/upgrading/agent-managed-updates.md), you can configure the update group so that you can control which instances get updated together.

**Dynamic configuration (recommended)**

To set the update group, edit the Discovery Config and set the `spec.gcp.install.update_group` key:

```
kind: discovery_config
# ...
spec:
  gcp:
   - install:
       update_group: "update-group-1"

```

**Static configuration**

To set the update group, specify the `install.update_group` key in your Discovery Service configuration:

```
# teleport.yaml
version: v3
# ...
discovery_service:
  enabled: true
  gcp:
   - install:
       update_group: "update-group-1"

```

### Configure HTTP Proxy during installation

For instances which require a proxy to access the installation files, you can configure HTTP Proxy settings in the Discovery Service.

**Dynamic configuration (recommended)**

To set the HTTP proxy settings, edit the Discovery Config and set the `spec.gcp.install.http_proxy_settings` key:

```
kind: discovery_config
# ...
spec:
  gcp:
   - install:
       http_proxy_settings:
         https_proxy: http://172.31.5.130:3128
         http_proxy: http://172.31.5.130:3128
         no_proxy: my-local-domain

```

**Static configuration**

You must set the `install.http_proxy_settings` key in your configuration:

```
# teleport.yaml
version: v3
# ...
discovery_service:
  enabled: true
  gcp:
   - install:
       http_proxy_settings:
         https_proxy: http://172.31.5.130:3128
         http_proxy: http://172.31.5.130:3128
         no_proxy: my-local-domain

```

### Use a custom installation script

To customize an installer, your user must have a role that allows `list`, `create`, `read` and `update` verbs on the `installer` resource.

Create a file called `installer-manager.yaml` with the following content:

```
kind: role
version: v5
metadata:
  name: installer-manager
spec:
  allow:
    rules:
      - resources: [installer]
        verbs: [list, create, read, update]

```

Create the role:

```
$ tctl create -f installer-manager.yaml
role 'installer-manager' has been created
```

---

TIP

You can also create and edit roles using the Web UI. Go to **Access -> Roles** and click **Create New Role** or pick an existing role to edit.

---

The preset `editor` role has the required permissions by default.

To customize the default installer script, execute the following command on your workstation:

```
$ tctl edit installer/default-installer
```

After making the desired changes to the default installer, save and close the file in your text editor.

Multiple `installer` resources can exist and be specified in the `gcp.install.script_name` section:

**Dynamic configuration (recommended)**

Edit the Discovery Config to specify a custom installer script:

```
kind: discovery_config
# ...
spec:
  gcp:
    - types: ["gce"]
      tags:
       - "env": "prod"
      install: # optional section when default-installer is used.
        script_name: "default-installer"
    - types: ["gce"]
      tags:
       - "env": "devel"
      install:
        script_name: "devel-installer"


```

**Static configuration**

Edit the `teleport.yaml` configuration to specify a custom installer script:

```
discovery_service:
  # ...
  gcp:
    - types: ["gce"]
      tags:
       - "env": "prod"
      install: # optional section when default-installer is used.
        script_name: "default-installer"
    - types: ["gce"]
      tags:
       - "env": "devel"
      install:
        script_name: "devel-installer"

```

---

The `installer` resource has the following templating options:

- `{{ .MajorVersion }}`: the major version of Teleport to use when installing from the repository.
- `{{ .PublicProxyAddr }}`: the public address of the Teleport Proxy Service to connect to.
- `{{ .RepoChannel }}`: Optional package repository (apt/yum) channel name. Has format `<channel>/<version>` e.g. stable/v19. See [installation](https://goteleport.com/docs/installation/linux.md) for more details.
- `{{ .AutomaticUpgrades }}`: indicates whether Automatic Updates are enabled or disabled. Its value is either `true` or `false`. See [Automatic Agent Updates](https://goteleport.com/docs/upgrading/agent-managed-updates.md) for more information.
- `{{ .TeleportPackage }}`: the Teleport package to use. Its value is either `teleport-ent` or `teleport` depending on whether the cluster is enterprise or not.

These can be used as follows:

```
kind: installer
metadata:
  name: default-installer
spec:
  script: |
    echo {{ .PublicProxyAddr }}
    echo Teleport-{{ .MajorVersion }}
    echo Repository Channel: {{ .RepoChannel }}
version: v1

```

Which, when retrieved for installation, will evaluate to a script with the following contents:

```
echo teleport.example.com
echo Teleport-19.0.0-dev
echo Repository Channel: stable/v19.0.0-dev

```

The default installer will take the following actions:

- Add an official Teleport repository to supported Linux distributions.
- Install Teleport via `apt` or `yum`.
- Generate the Teleport config file and write it to `/etc/teleport.yaml`.
- Enable and start the Teleport service.

## Next steps

- Read [Joining Services via GCP](https://goteleport.com/docs/enroll-resources/agents/gcp.md) for more information on GCP tokens.
- Full documentation on GCP discovery configuration can be found through the [config file reference documentation](https://goteleport.com/docs/reference/deployment/config.md).