How Active Directory Replication Topology Works.doc

How Active Directory Replication Topology WorksHow Active Directory Replication Topology WorksIn this section Active Directory KCC Architecture and Processes Replication Topology Physical Structure Performance Limits for Replication Topology Generation Goals of Replication Topology Topology-Related Objects in Active Directory Replication Transports Replication Between Sites KCC and Topology Generation Network Ports Used by Replication Topology Related InformationActive Directory implements a replication topology that takes advantage of the network speeds within sites, which are ideally configured to be equivalent to local area network (LAN) connectivity (network speed of 10megabits per second Mbps or higher). The replication topology also minimizes the use of potentially slow or expensive wide area network (WAN) links between sites.When you create a site object in Active Directory, you associate one or more Internet Protocol (IP) subnets with the site. Each domain controller in a forest is associated with an Active Directory site. A client workstation is associated with a site according to its IP address; that is, each IP address maps to one subnet, which in turn maps to one site.Active Directory uses sites to: Optimize replication for speed and bandwidth consumption between domain controllers. Locate the closest domain controller for client logon, services, and directory searches. Direct a Distributed File System (DFS) client to the server that is hosting the requested data within the site. Replicate the system volume (SYSVOL), a collection of folders in the file system that exists on each domain controller in a domain and is required for implementation of Group Policy.The ideal environment for replication topology generation is a forest that has a forest functional level of Windows Server2003. In this case, replication topology generation is faster and can accommodate more sites and domains than occurs when the forest has a forest functional level of Windows2000. When at least one domain controller in each site is running Windows Server2003, more domain controllers in each site can be used to replicate changes between sites than when all domain controllers are running Windows2000 Server.In addition, replication topology generation requires the following conditions: A Domain Name System (DNS) infrastructure that manages the name resolution for domain controllers in the forest. Active Directoryintegrated DNS is assumed, wherein DNS zone data is stored in Active Directory and is replicated to all domain controllers that are DNS servers. All physical locations that are represented as site objects in Active Directory have LAN connectivity. IP connectivity is available between each site and all sites in the same forest that host operations master roles. Domain controllers meet the hardware requirements for Microsoft Windows Server2003, Standard Edition; Windows Server2003, Enterprise Edition; and Windows Server2003, Datacenter Edition. The appropriate number of domain controllers is deployed for each domain that is represented in each site.This section covers the replication components that create the replication topology and how they work together, plus the mechanisms and rationale for routing replication traffic between domain controllers in the same site and in different sites.Active Directory KCC Architecture and ProcessesThe replication topology is generated by the Knowledge Consistency Checker (KCC), a replication component that runs as an application on every domain controller and communicates through the distributed Active Directory database. The KCC functions locally by reading, creating, and deleting Active Directory data. Specifically, the KCC reads configuration data and reads and writes connection objects. The KCC also writes local, nonreplicated attribute values that indicate the replication partners from which to request replication.For most of its operation, the KCC that runs on one domain controller does not communicate directly with the KCC on any other domain controller. Rather, all KCCs use the knowledge of the common, global data that is stored in the configuration directory partition as input to the topology generation algorithm to converge on the same view of the replication topology.Each KCC uses its in-memory view of the topology to create inbound connections locally, manifesting only those results that apply to itself. The KCC communicates with other KCCs only to make a remote procedure call (RPC) request for replication error information. The KCC uses the error information to identify gaps in the replication topology. A request for replication error information occurs only between domain controllers in the same site.Note The KCC uses only RPC to communicate with the directory service. The KCC does not use Lightweight Directory Access Protocol (LDAP).One domain controller in each site is selected as the Intersite Topology Generator (ISTG). To enable replication across site links, the ISTG automatically designates one or more servers to perform site-to-site replication. These servers are called bridgehead servers. A bridgehead is a point where a connection leaves or enters a site.The ISTG creates a view of the replication topology for all sites, including existing connection objects between all domain controllers that are acting as bridgehead servers. The ISTG then creates inbound connection objects for servers in its site that it determines will act as bridgehead servers and for which connection objects do not already exist. Thus, the scope of operation for the KCC is the local server only, and the scope of operation for the ISTG is a single site.Each KCC has the following global knowledge about objects in the forest, which it gets by reading objects in the Sites container of the configuration directory partition and which it uses to generate a view of the replication topology: Sites Servers Site affiliation of each server Global catalog servers Directory partitions stored by each server Site links Site link bridgesDetailed information about these configuration components and their functionality is provided later in this section.The following diagram shows the KCC architecture on servers in the same forest in two sites.KCC Architecture and ProcessesThe architecture and process components in the preceding diagram are described in the following table.KCC Architecture and Process ComponentsComponent Description Knowledge Consistency Checker (KCC)The application running on each domain controller that communicates directly with the Ntdsa.dll to read and write replication objects.Directory System Agent (DSA)The directory service component that runs as Ntdsa.dll on each domain controller, providing the interfaces through which services and processes such as the KCC gain access to the directory database.Extensible Storage Engine (ESE)The directory service component that runs as Esent.dll. ESE manages the tables of records, each with one or more columns. The tables of records comprise the directory database.Remote procedure call (RPC)The Directory Replication Service (Drsuapi) RPC protocol, used to communicate replication status and topology to a domain controller. The KCC also uses this protocol to communicate with other KCCs to request error information when building the replication topology.Intersite Topology Generator (ISTG)The single KCC in a site that manages intersite connection objects for the site.The four servers in the preceding diagram create identical views of the servers in their site and generate connection objects on the basis of the current state of Active Directory data in the configuration directory partition. In addition to creating its view of the servers in its respective site, the KCC that operates as the ISTG in each site also creates a view of all servers in all sites in the forest. From this view, the ISTG determines the connections to create on the bridgehead servers in its own site.Note A connection requires two endpoints: one for the destination domain controller and one for the source domain controller. Domain controllers creating an intrasite topology always use themselves as the destination end point and must consider only the endpoint for the source domain controller. The ISTG, however, must identify both endpoints in order to create connection objects between two other servers.Thus, the KCC creates two types of topologies: intrasite and intersite. Within a site, the KCC creates a ring topology by using all servers in the site. To create the intersite topology, the ISTG in each site uses a view of all bridgehead servers in all sites in the forest. The following diagram shows a high-level generalization of the view that the KCC sees of an intrasite ring topology and the view that the ISTG sees of the intersite topology. Lines between domain controllers within a site represent inbound and outbound connections between the servers. The lines between sites represent configured site links. Bridgehead servers are represented as BH.KCC and ISTG Views of Intrasite and Intersite TopologyReplication Topology Physical StructureThe Active Directory replication topology can use many different components. Some components are required and others are not required but are available for optimization. The following diagram illustrates most replication topology components and their place in a sample Active Directory multisite and multidomain forest. The depiction of the intersite topology that uses multiple bridgehead servers for each domain assumes that at least one domain controller in each site is running Windows Server2003. All components of this diagram and their interactions are explained in detail later in this section.Replication Topology Physical StructureIn the preceding diagram, all servers are domain controllers. They independently use global knowledge of configuration data to generate one-way, inbound connection objects. The KCCs in a site collectively create an intrasite topology for all domain controllers in the site. The ISTGs from all sites collectively create an intersite topology. Within sites, one-way arrows indicate the inbound connections by which each domain controller replicates changes from its partner in the ring. For intersite replication, one-way arrows represent inbound connections that are created by the ISTG of each site from bridgehead servers (BH) for the same domain (or from a global catalog server GC acting as a bridgehead if the domain is not present in the site) in other sites that share a site link. Domains are indicated as D1, D2, D3, and D4. Each site in the diagram represents a physical LAN in the network, and each LAN is represented as a site object in Active Directory. Heavy solid lines between sites indicate WAN links over which two-way replication can occur, and each WAN link is represented in Active Directory as a site link object. Site link objects allow connections to be created between bridgehead servers in each site that is connected by the site link. Not shown in the diagram is that where TCP/IP WAN links are available, replication between sites uses the RPC replication transport. RPC is always used within sites. The site link between SiteA and SiteD uses the SMTP protocol for the replication transport to replicate the configuration and schema directory partitions and global catalog partial, read-only directory partitions. Although the SMTP transport cannot be used to replicate writable domain directory partitions, this transport is required because a TCP/IP connection is not available between SiteA and SiteD. This configuration is acceptable for replication because SiteD does not host domain controllers for any domains that must be replicated over the site link A-D.By default, site links A-B and A-C are transitive (bridged), which means that replication of domain D2 is possible between SiteB and SiteC, although no site link connects the two sites. The cost values on site links A-B and A-C are site link settings that determine the routing preference for replication, which is based on the aggregated cost of available site links. The cost of a direct connection between SiteC and SiteB is the sum of costs on site links A-B and A-C. For this reason, replication between SiteB and SiteC is automatically routed through SiteA to avoid the more expensive, transitive route. Connections are created between SiteB and SiteC only if replication through SiteA becomes impossible due to network or bridgehead server conditions.Performance Limits for Replication Topology GenerationActive Directory topology generation performance is limited primarily by the memory on the domain controller. KCC performance degrades at the physical memory limit. In most deployments, topology size will be limited by the amount of domain controller memory rather than CPU utilization required by the KCC.Scaling of sites and domains is improved in Windows Server2003 by improving the algorithm that the KCC uses to generate the intersite replication topology. Because all domain controllers must use the same algorithm to arrive at a consistent view of the replication topology, the improved algorithm has a forest functional level requirement of Windows Server2003 or Windows Server2003 interim.KCC scalability was tested on domain controllers with 1.8GHz processor speed, 512megabytes (MB) RAM, and small computer system interface (SCSI) disks. KCC performance results at the Windows Server2003 forest functional level are described in the following table. The times shown are for the KCC to run where all new connections are needed (maximum) and where no new connections are needed (minimum). Because most organizations add domain controllers in increments, the minimum generation times shown are closest to the actual runtimes that can be expected in deployments of comparable sizes. The CPU and memory usage values for the Local Security Authority (LSA) process (Lsass.exe) indicate the more significant impact of memory versus percent of CPU usage when the KCC runs.Note Active Directory runs as part of the LSA, which manages authentication packages and authenticates users and services.Minimum and Maximum KCC Generation Times for Domain-Site CombinationsDomains Sites Connections KCC Generation Time (seconds) Lsass.exe Memory Usage (MB) Lsass.exe CPU Usage (%) 1500Maximum4310039Minimum1100291,000Maximum4914943Minimum2149283,000Maximum6923646Minimum2236635500Maximum7012529Minimum2126711,000Maximum7723728Minimum3237782,000Maximum7832543Minimum5325773,000Maximum8544952Minimum6449754,000Maximum55562446Minimum3462469201,000Maximum4842365Minimum542381401,000Maximum9379956Minimum12799962,000Minimum3887471These numbers cannot be used as the sole guidelines for forest and domain design. Other limitations might affect performance and scalability. A limitation of note is that when FRS is deployed, a limit of 1,200 domain controllers per domain is recommended to ensure reliable recovery of SYSVOL.For more information about FRS limitations, see “FRS Technical Reference.” For more information about the functional level requirements for the intersite topology generation algorithm, see “Automated Intersite Topology Generation” later in this section.Goals of Replication TopologyThe KCC generates a replication topology that achieves the following goals: Connect every directory partition replica that must be replicated. Control replication latency and cost. Route replication between sites. Effect client affinity.By default, the replication topology is managed automatically and optimizes existing connections. However, manual connections created by an administrator are not modified or optimized.Connect Directory Partition ReplicasThe total replication topology is actually composed of several underlying topologies, one for each directory partition. In the case of the schema and configuration directory partitions, a single topology is created. The underlying topologies are merged to form the minimum number of connections that are required to replicate each directory partition between all domain controllers that store replicas. Where the connections for directory partitions are identical between domain controllers for example, two domain controllers store the same domain directory partition a single connection can be used for replication of updates to the domain, schema, and configuration directory partitions.A separate replication topology is also created for application directory partitions. However, in the same manner as schema and configuration directory partitions, application directory partitions can use the same topology as domain directory partitions. When application and domain directory partitions are common to the source and destination domain controllers, the KCC does not create a separate connection for the application directory partition.A separate topology is not created for the partial replicas that are stored on global catalog servers. The connections that are needed by a global catalog server to replicate each partial replica of a domain are part of the topology that is created for each domain.The routes for the following directory partitions or combinations of directory partitions are aggregated to arrive at the overall topology: Configuration and schema within a site. Each writable domain directory partition within a site. Each application directory partition within a site. Global catalog read-only, partial domain di