Before you set up an Exadata Cloud Service instance, you must set up a virtual cloud network (VCN) and other Networking service components. This topic describes the recommended configuration for the VCN and several related requirements for the Exadata Cloud Service instance.
VCN and Subnets
To launch an Exadata Cloud Service instance, you must have:
- A VCN in the region where you want the Exadata Cloud Service instance
At least two subnets in the VCN. The two subnets are:
- Client subnet
- Backup subnet
For Exadata Cloud Service instances using the new Exadata resource model, networking is configured on the cloud VM cluster resource. For instances using DB system resource model, networking is configured on the DB system resource.
In general, Oracle recommends using regional subnets , which span all availability domains in the region. If you instead use AD-specific subnets , both the client and backup subnets must be in the same availability domain. The important thing to know for your Exadata Cloud Service instance is that the resources you create in the two subnets must be in the same availability domain. For more information, see Overview of VCNs and Subnets.
You will create custom route tables for each subnet. You will also create security rules to control traffic to and from the client network and backup network of the Exadata compute notes (for cloud VM clusters, nodes are called virtual machines). More information follows about those items.
Option 1: Public Client Subnet with Internet Gateway
This option can be useful when doing a proof-of-concept or development work. You can use this setup in production if you want to use an internet gateway with the VCN, or if you have services that run only on a public network and need access to the database. See the following diagram and description.
You set up:
- Public client subnet (public means that the resources in the subnet can have public IP addresses at your discretion).
- Private backup subnet (private means that the resources in the subnet cannot have public IP addresses and therefore cannot receive incoming connections from the internet).
Gateways for the VCN:
- Internet gateway (for use by the client subnet).
- Service gateway (for use by the backup subnet). Also see Option 1: Service Gateway Access Only to Object Storage.
- Custom route table for the public client subnet, with a route for 0.0.0.0/0, and target = the internet gateway.
- Separate custom route table for the private backup subnet, with a route rule for the service CIDR label called OCI <region> Object Storage, and target = the service gateway. Also see Option 1: Service Gateway Access Only to Object Storage.
- Security rules to enable the desired traffic to and from the Exadata virtual machines compute nodes. See Security Rules for the Exadata Cloud Service instance.
- Static route on the Exadata Cloud Service instance's compute nodes (to enable access to Object Storage by way of the backup subnet).
See this known issue for information about configuring route rules with service gateway as the target on route tables associated with public subnets.
Option 2: Private Subnets
Oracle recommends this option for a production system. Both subnets are private and cannot be reached from the internet. See the following diagram and description.
You set up:
- Private client subnet.
- Private backup subnet.
Gateways for the VCN:
- Dynamic routing gateway (DRG), with a FastConnect or Site-to-Site VPN to your on-premises network (for use by the client subnet).
- Service gateway (for use by the backup subnet to reach Object Storage, and for use by the client subnet to reach the Oracle YUM repository for OS updates). Also see Option 2: Service Gateway Access to Both Object Storage and YUM Repos.
- NAT gateway (for use by the client subnet to reach public endpoints not supported by the service gateway).
Custom route table for the private client subnet, with two rules:
- A rule for the on-premises network's CIDR, and target = DRG.
- A rule for the service CIDR label called All <region> Services in Oracle Services Network, and target = the service gateway. The Oracle Services Network is a conceptual network in Oracle Cloud Infrastructure that is reserved for Oracle services. The rule enables the client subnet to reach the regional Oracle YUM repository for OS updates. Also see Option 2: Service Gateway Access to Both Object Storage and YUM Repos.
- A rule for 0.0.0.0/0, and target = NAT gateway.
- Separate custom route table for the private backup subnet, with one rule:
- The same rule as for the client subnet: for the service CIDR label called All <region> Services in Oracle Services Network, and target = the service gateway. This rule enables the backup subnet to reach the regional Object Storage for backups.
- Security rules to enable the desired traffic to and from the Exadata nodes. See Security Rules for the Exadata Cloud Service instance.
- Static route on the compute nodes (for VM clusters, the virtual machines) to enable access to Object Storage by way of the backup subnet.
Requirements for IP Address Space
If you're setting up Exadata Cloud Service instances (and thus VCNs) in more than one region, make sure the IP address space of the VCNs does not overlap. This is important if you want to set up disaster recovery with Oracle Data Guard.
The two subnets you create for the Exadata Cloud Service instance must not overlap with 192.168.128.0/20.
The following table lists the minimum required subnet sizes, depending on the Exadata rack size. For the client subnet, each node requires three IP addresses, and in addition, three addresses are reserved for Single Client Access Names (SCANs). For the backup subnet, each node requires two addresses.
The Networking service reserves three IP addresses in each subnet. Allocating a larger space for the subnet than the minimum required (for example, at least /25 instead of /28) can reduce the relative impact of those reserved addresses on the subnet's available space.
| Rack Size | Client Subnet: # Required IP Addresses | Client Subnet: Minimum Size | Backup Subnet: # Required IP Addresses | Backup Subnet: Minimum Size |
|---|---|---|---|---|
| Base System or Quarter Rack | (3 addresses * 2 nodes) + 3 for SCANs + 3 reserved in subnet = 12 | /28 (16 IP addresses) | (2 address * 2 nodes) + 3 reserved in subnet = 7 | /29 (8 IP addresses) |
| Half Rack | (3 * 4 nodes) + 3 + 3 = 18 | /27 (32 IP addresses) | (2 * 4 nodes) + 3 = 11 | /28 (16 IP addresses) |
| Full Rack | (3* 8 nodes) + 3 + 3 = 30 | /27 (32 IP addresses) | (2 * 8 nodes) + 3 = 19 | /27 (32 IP addresses) |
| Flexible infrastructure systems (X8M and higher) | 3 addresses per database node (minimum of 2 nodes) + 3 for SCANs + 3 reserved in subnet | Minimum size determined by total number of IP addresses needed for database nodes | 2 address per database node (minimum of 2 nodes) + 3 reserved in subnet | Minimum size determined by total number of IP addresses needed for database nodes |
VCN Creation Wizard: Not for Production
The Networking section of the Console includes a handy wizard that creates a VCN along with related resources. It can be useful if you just want to try launching an instance. However, the wizard automatically creates a public subnet and an internet gateway. You may not want this for your production network, so Oracle recommends you create the VCN and other resources individually yourself instead of using the wizard.
DNS: Short Names for the VCN, Subnets, and Exadata Cloud Service instance
For the nodes to communicate, the VCN must use the Internet and VCN Resolver. It enables hostname assignment to the nodes, and DNS resolution of those hostnames by resources in the VCN. It enables round robin resolution of the database's SCANs. It also enables resolution of important service endpoints required for backing up databases, patching, and updating the cloud tooling on an Exadata Cloud Service instance. The Internet and VCN Resolver is the VCN's default choice for DNS in the VCN. For more information, see DNS in Your Virtual Cloud Network and also DHCP Options.
When you create the VCN, subnets, and Exadata, you must carefully set the following identifiers, which are related to DNS in the VCN:
- VCN domain label
- Subnet domain label
- Hostname prefix for the Exadata Cloud Service instance's cloud VM cluster or DB system resource
These values make up the node's fully qualified domain name (FQDN):
<hostname_prefix>-######.<subnet_domain_label>.<vcn_domain_label>.oraclevcn.com
For example:
exacs-abcde1.clientpvtad1.acmevcniad.oraclevcn.com
In this example, you assign exacs as the hostname prefix when you create the cloud VM cluster or DB system. The Database service automatically appends a hyphen and a five-letter string with the node number at the end. For example:
- Node 1:
exacs-abcde1.clientpvtad1.acmevcniad.oraclevcn.com - Node 2:
exacs-abcde2.clientpvtad1.acmevcniad.oraclevcn.com - Node 3:
exacs-abcde3.clientpvtad1.acmevcniad.oraclevcn.com - And so on
Requirements for the hostname prefix:
- Recommended maximum: 12 characters. For more information, see the example under the following section, "Requirements for the VCN and subnet domain labels".
- Cannot be the string localhost
Requirements for the VCN and subnet domain labels:
- Recommended maximum: 14 characters each. The actual underlying requirement is a total of 28 characters across both domain labels (excluding the period between the labels). For example, both of these are acceptable:
subnetad1.verylongvcnphxorverylongsubnetad1.vcnphx. For simplicity, the recommendation is 14 characters each. - No hyphens or underscores.
Recommended: include the region name in the VCN's domain label, and include the availability domain name in the subnet's domain label.
In general, the FQDN has a maximum total limit of 63 characters. Here is a safe general rule:
<12_chars_max>-######.<14_chars_max>.<14_chars_max>.oraclevcn.com
The preceding maximums are not enforced when you create the VCN and subnets. However, if the labels exceed the maximum, the Exadata deployment fails.
DNS: Between On-Premises Network and VCN
Node Access to Object Storage: Static Route
To be able to back up databases, and patch and update cloud tools on an Exadata Cloud Service instance, you must configure access to Oracle Cloud Infrastructure Object Storage. Regardless of how you configure the VCN with that access (for example, with a service gateway), you may also need to configure a static route to Object Storage on each of the compute nodes in the cluster. This is only required if you are not using automatic backups. If you are using customized backups using the backup APIs, then you must route traffic destined for Object Storage through the backup interface (BONDETH1). This is not necessary if you are using the automatic backups created with the Console, APIs, or CLIs.
You must configure a static route for Object Storage access on each compute node in an Exadata Cloud Service instance if you are not creating automatic backups with the Console, APIs, or CLIs. Otherwise, attempts to back up databases, and patch or update tools on the system, can fail.
Service Gateway for the VCN
Security Rules for the Exadata Cloud Service instance
This section lists the security rules to use with your Exadata Cloud Service instance. Security rules control the types of traffic allowed for the client network and backup network of the Exadata's compute nodes. The rules are divided into three sections.
There are different ways to implement these rules. For more information, see Ways to Implement the Security Rules.
Rules Required for Both the Client Network and Backup Network
This section has several general rules that enable essential connectivity for hosts in the VCN.
If you use security lists to implement your security rules, be aware that the rules that follow are included by default in the default security list. Update or replace the list to meet your particular security needs. The two ICMP rules (general ingress rules 2 and 3) are required for proper functioning of network traffic within the Oracle Cloud Infrastructure environment. Adjust the general ingress rule 1 (the SSH rule) and the general egress rule 1 to allow traffic only to and from hosts that require communication with resources in your VCN.
Rules Required Specifically for the Client Network
The following security rules are important for the client network.
- For X8M systems, Oracle recommends that all ports on the client subnet need to be open for ingress and egress traffic. This is a requirement for adding additional database servers to the system.
- Client ingress rules 1 and 2 only cover connections initiated from within the client subnet. If you have a client that resides outside the VCN, Oracle recommends setting up two additional similar rules that instead have the Source CIDR set to the public IP address of the client.
- Client ingress rules 3 and 4 and client egress rules 1 and 2 allow TCP and ICMP traffic inside the client network and enable the nodes to communicate with each other. If TCP connectivity fails across the nodes, the Exadata cloud VM cluster or DB system resource fails to provision.
Rule Required Specifically for the Backup Network
The following security rule is important for the backup network because it enables the DB system to communicate with Object Storage through the service gateway (and optionally with the Oracle YUM repos if the client network doesn't have access to them). It is redundant with the general egress rule in Security Rules for the Exadata Cloud Service instance (and in the default security list). It is optional but recommended in case the general egress rule (or default security list) is inadvertently changed.
Ways to Implement the Security Rules
The Networking service offers two ways to implement security rules within your VCN:
For a comparison of the two methods, see Comparison of Security Lists and Network Security Groups.
No comments:
Post a Comment