Symmetrix DMX系列存储深入介绍VIP

1、Symmetrix DMX系列存储深入介绍330331DMX800DMX1000DMX2000Symmetrix DMX SeriesDMX3000332ESCON Fibre channel (futures multi-protocol director) SCSI, ESCON Fibre Channel, FICON, Multi-protocol directorHost SupportCCM (w/ processor & redundant)ENET Hub on doorCommunications Utilities16 GB/s1.6 GB/sSystem Aggregat

2、e Bandwidth (max)250 MB/s150 MB/sPer Port Bandwidth (actual)Point-to-pointMulti-drop busCache InterconnectTop/bottomBottomHost Cable Management11PCECM (w/ processor & redundant)EPO Box (no processor)Environmental Utilities12V5V/12VPower Distribution2(N+1) (scalable)N+1Power Redundancy288384Disks ( m

3、ax)FibreChannel-Arbitrated LoopSCSI LVDDisk Type84Memory slots24 slots20 slotsMain Card Cage2-bay3-bayCabinet DMX2000Symm 8830 8000 Series vs. DMX Series 333334Architectural Overview DMX = Direct Matrix Architecture and refers to the improved Director to cache connectivity. DFs (DAs in Fibre) back e

4、nd Directors and FAs front end Directors are now the same physical board running different Emulation code thus reducing Manufacturing reliability and testing cost.335DMX2000 Front ViewThe DMX2000 has a 24-slot card cage, 8 Slots are reserved for memory only, the other 16 slots are dedicated to Host

5、Adapters and Disk Directors.DMX2000 uses a 2-zone power distribution strategy, where Zone-A powers one half of the system, and Zone-B powers the other half. Memory is powered by both zones. Power System is scalable; Add power as needed (upgrades in minimum of 2 power supplies, due to zoning)Up to 28

6、8 drives; each midplane holds up to 4x 36drives = 144 drives in the front and144 drives in the rear.2 x PLIM ( Power Line Input Modules ). Each AC-power input cable has its own PLIM. Cooling unit contains 6x Dual fan trays. Zoning applies to fan trays as well.336DMX2000 Rear ViewDMX2000 allows for C

7、able routing via the top of the cabinet as well as the bottom Cable access panelThe rear card cage holds the host/disk adapter cards as well as the Communications Control Modules (CCM), and Environmental Control Modules (ECM), which reside behind the Memory boards.DMX2000 deploys 2 battery (172 poun

8、ds each), the DMX1000 has only one, they are the same battery in both337DMX2000 - Slot Configuration (Front)DIR1Slot0BEDIR2Slot1BEDIR8Slot7FEDIR3Slot2FEDIR4Slot3FEDIR5Slot4BEor FEDIR6Slot5BEor FEDIR7Slot6FEDIR9Slot8FEDIR10Slot9FEDIR16SlotFBEDIR11SlotABEor FEDIR12SlotBBEor FEDIR13SlotCFEDIR14SlotDFED

9、IR15SlotEBEM2Slot12M0Slot10M4Slot14M5Slot15M6Slot16M7Slot17M3Slot13M1Slot11EMC recommends channel directors be installed in pairs for redundancy. Channel directors are the same (interchangeable) for DMX1000 and DMX2000.Channel directors can be intermixed in any combination of Fibre Channel and ESCON

10、 in the DMX1000 and DMX2000.338 Matrix ArchitectureEach memory board has sixteen ports, one to each director. Each region can sustain a data rate of 500MBs 4 per card so 2GB per card. If a director is removed from a system, the usable bandwidth is not reduced.If a memory board is removed the usable

11、bandwidth is dropped by 2GB/s339Communications and EnvironmentalsDirector to Director Communications will bypass the data path via the CCM Communication Control Module, A high-performance network interface to each of the Power PC CPUs on the director boards. CCM carries control traffic only, bypassi

12、ng cache. Environmental monitoring is a function of ECM Environmental Control Module.340DirectorsMemory BoardsPort Bypass CardsFront PanelCabinet door switchesRed/Black switchFans topEPI PortsEPI Selector SwitchAC Input ModulePower SuppliesBattery Box/ChargersSmoke Detector UnitCCM CommInter-ECM Com

13、m.PCExpansion Ports, Spares, misc.System IDECMECMThe ECM (Environmental Control Module) handles all maintenance and control of the environmental functionsMonitors and logs environmental events, reports any operational problems, such as thermal excursion, voltage droop, etc. Monitors temperature, vol

14、tage, and fan speed remotely via the PICThere are 2 ECM boards installed per system. Under normal conditions ECM0 will be active, whereas ECM1 will remain in a dormant state until ECM0 fails. 341DMX2000 Disk BayEight (8) DiskCages with 36LP Disks MaxIn Each (288)B1B2B3B4F1F3F4F2DMX2000 has 8 Disk ca

15、ges; four (4) in the front and four in the rear, each contains up to 36 Drives and 2 PBCs (port bypass cards). Each PBC can access all 36 Disks in the Disk cage.342The PBC contains the switch elements and control functions to allow intelligent management of the two FC-AL loops embedded in each disk

16、midplane. There are two PBCs per disk midplane. Each disk midplane can support 36 FC drives.343Arbitrated LoopIf the director wants to talk to one of the disks,ALPA C9, then the director first needs to arbitrate to gain control of the loop, and needs to open communications with ALPA C9. Frames trave

17、l in one direction around the loop. If a frame is destined for ALPA C9 from the Disk director (ALPA 01), the first node which will see the frame is C5. It will see that the frame is not for it and will pass it on. ALPA B5 will then see that the frame is not for it and will pass it on until eventuall

18、y C9 will receive the frame. It should be noted here that obviously the more disks (nodes) which are put on a loop, the slower the loop will become. Hence we have the 9 disk loop model of Symm-6 being the performance machine as opposed to the standard 18 disk loop model.344PBC Redundant Configuratio

19、n16aC3AB16aC1AB1aC2AB1aC0ABPortBypassDisk Director 16aAPortBypassSeparate Physical PathsDisk Director 1aAStar topology gives us 2 access paths to each drive using the Port Bypass Card ( all drives are dual ported )You can see from this slide that Controller redundancy is now across Directors 1a and

20、16a.345DMX Drive SubsystemThe Disk Subsystem for the DMX has greatly improved, this was achieved by moving from a SCSI Bus Architecture to a Dual Fibre Channel Arbitrated Loop (FCAL) systemThe Port Bypass card creates the loop for us346DMX Backend Solution DMX1000/DMX2000347DMX2000-P Disk map showin

21、g 9 drive loop, note all disks in the front are being accessed by Directors A ports, also notice a and b processor or c and d processor are in the same disk cage.DMX2000-P FrontP Performance (9 drives per loop)S Standard (18 drives per loop )348Power Rack BPower Rack AMainAux.ABPowerDMX2000 Power Ra

22、ck Front View of Power Racks and PLIMPower system has 2 Racks w/6 supply capacity & 1 PLIM (Power Line Input Module) shelfMain to Aux sensed & switched within each moduleFour line cords per cabinet (1 main & 1 aux per PLIM)Two versions: Single phase and three phase . PS9PS10PS11PS6PS7PS8PS3PS4PS5PS0

23、PS1PS2Front ViewAB349AC Main “B”AC Aux “B”AC to Rack B12V Bus Bars56V Bus BarsAC Main “A”AC Aux “A”AC to Rack ARear View of Power Racks and PLIMThe cabinet has four power cords: one main and one auxiliary AC line cord per PLIM. The power supplies are also able to operate from 56V battery backup. A 5

24、6V output will charge the battery; the 56V output is full current sharing to battery, charger, and electronics residing in battery box. Fan modules are AC input with local AC/DC converter onboard that power DC fans.350351Power Zone APower Zone A+BPower Zone BFRONTREARDMX2000 Power Zones352 DMX100035

25、3Front and Rear Internal View354DIR1Slot0BEDIR2Slot1BEor FEDIR3Slot2FEDIR4Slot3FEDIR16SlotFBEDIR13SlotCFEDIR14SlotDFEDIR15SlotEBEor FEM0Slot10M6Slot16M7Slot17M1Slot11DMX1000 - Slot Configuration (Front)The DMX1000 system has a 12-slot backplane. On the front side, the four (4) slots in the center of

26、 the backplane are reserved for memory and the remaining eight (8) slots are reserved for directors (4 to the left and 4 to the right of memory).Dir 116 reside in Slot 0F, Memory boards reside in: Slot 10,11,16 and 17355Distributed Matrix The DMX1000 system has eight directors and four memory boards

27、. The system bandwidth is defined by the number of memory boards and, since there is half the number of memory boards, the bandwidth is half that of the DMX2000 system at 8 GB/s. Notice that the architecture has also changed in that each director now has two channels to each memory board.356CCMECMAd

28、apter TerminatorAdapter slots1.3.4.2.3.4.Midplane - RearCommunications and Environmental Control ModulesThe CCM (Communications Control Module) handles all maintenance and control of the communications functions in the DMX Series system. The ECM (Environmental Control Module) handles all maintenance

29、 and control of the environmental functions in the DMX Series system. The CCM and ECM also include an onboard microprocessor which allows the maintenance programs to be embedded on each of the boards. 357ECM, CCM, & InlinesChecking for power or other environmental issues is done from the ECM,CCM tab

30、 358Inlines from ECMHELPShow Help displayVERSIONDisplay CCM versionB3List FRUs with a problemB3,LISTList all the FRUsB3,Information on FRU , like voltagesB8,VPD informationEB,Works the same as on a director359Lists all the Field Replaceable Units in the Symm 360DMX800RMS 361RMS Definition of TermsSt

31、orage Processor Enclosure (SPE ) Symmetrix component of system with DMX director and memory boardsDAE Disk Array Enclosure; designed by Clariion and holds 15 Fibre Channel drivesKatana Clariion code name for DAEFEBE Front End / Back End board; combination of back adapters and DMX ECM/CCMSPS Standby

32、Power Supply; equivalent to Symm6 batteryLCC Link Control Card; equivalent to Symm6 PBC362 DMX800DMX800 is shown here with two SPE (Storage Processor Enclosure) each containing 4 directors, 2 memory boards, 2 FEBE (front-end/back-end) boards, Two 12V power supplies (PS1 & PS2) One fan module . KVM -

33、 Keyboard, View Screen & Mouse is an optional extra, management of DMX800 will normally be via CE laptop to a 1u server. DAE (Disk Array Enclosure) Contains 15 Fibre disk drives, 2 power supplies, 2 LCC (Link Control Card)1 SPS enclosure (2 SPS) (Standby Power Supply ) provides battery backup to SPE

34、 (battery on for only 90 seconds) SPS uses RS232 communication. 1 SPS enclosure (2 SPS) are provided per 2 DAE .363DMX800 Storage Processor EnclosureHere is an exploded view of the SPE (Storage Processor Enclosure).4 directors2 memory boards2 FEBE (front-end/back-end) boardsTwo 12V power suppliesOne

35、 fan module364DMX800 Card CageHere the Director numbers are shown, A maximum of 4 Directors are possible, Dir 1,2 15 and 16 with two Memory Boards and 2 FEBE front end back end boards.365DMX-800 - SPE EnclosureSlot ID1ASlot ID 0Slot ID 1Slot ID 10Slot ID 11Slot ID ESlot ID 1BSlot ID FDMX800 introduc

36、es the first FEBE (Front-End/Back-End) board, which serves to replace the individual adapters by providing port connections to both the Hosts (HBAs) and backend (disks). The DMX800 contains two FEBE boards (F0 & F1), each with 16 ports. The FEBE ports are divided with half of each board supporting t

37、he front-end and half supporting the back-end. 366FEBE Functions Termination Environmental Control Ethernet SwitchHandles Fibre Channel Back End Adapter Handles Fibre Channel Front End AdapterEach FEBE board will support a maximum of 16 FC ports Optical Small Form Plug-in (SFP) modulesNote: FEBE act

38、s as an Adapter, ECM & CCM.FEBE has limited ECM / CCM functionality.367BackEndDIRFEBE DAE CACHEFrontEndDIRFEBEdBcbaABABABAdBcbaABABABAdBcbaABABABAdBcbaABABABADir 1Dir 2Dir 15Dir 16CacheD1 D0 C1 C0 B1 B0 A1 A0 B1 B0 A1 A0D1 D0 C1 C0D1 D0 C1 C0 B1 B0 A1 A0 D1 D0 C1 C0 B1 B0 A1 A0FEBE 0FEBE 1Half of th

39、e ports are for the front end and half are for the back end368DMX800 Fan & Power Supply IndicatorsThe Front view of the DMX800 SPE (Storage Processor Enclosure) has a hot swappable Fan module which consists of 3 fans, any 2 of which provide sufficient cooling, Fan module has a general fault LED indi

40、cator. Air flow is from front to rear of unit . There are Power On LED indicators on the SPE for 12V. PS1 and PS2 also have Power and Fault LED369Disk Array Enclosure (DAE)Each DAE Disk Array Enclosure holds up to 15 Fibre Channel drives (minimum 4 per DAE), each disk has a green and amber LED, the

41、green LED will flicker on off to indicate disk activity, the amber LED if lit indicates a fault. The DAE itself also has two LED to Indicate both power supplys are functioning normally.370 DAE rear viewLink Control Card ALink Control Card BThe DAE also known as Katana contains 2 Power Supplies 2 Lin

42、k Control cards up to 15 disks and four cooling fans.Each of these 2 power supplies will plug into an SPS which will in turn plug into the left or right PDU.When a Power supply is replaced, the CE has 3 minutes to complete the swap or all the drives in the DAE will go not ready.371Back-End - Drives7

43、3 GB 10krpmDisk AssemblyThe Symmetrix DMX Series supports high performance 10,000 RPM Fibre Channel hard drives in 73 GB capacity and 146 GB/10krpm Fibre Channel drives. Future support includes 73 GB/15krpm. Physical positioning of the drives in the DMX1000 and DMX2000 systems is designated by Symmw

44、in only, therefore not manually. Drives are populated based upon disk director and back-end fibre loop configurations. 372Link Control CardThe LCCs are connected to FEBE ports via twin-axial copper cables.The cabling is not explicitly configured as a loop, but as a set offull-duplex, point-to-point

45、connections, as shown below.373SPS Modules LayoutEach SPS has 1 AC inlet and 2 AC outlets with 5 internal batteries. SPS normally provides AC to two DAE if AC power is lost SPS can sustain Power ( to SPE or DAE ) for 90 - 120 seconds. Note: SPS for DAE and DMX800 are identical units. Due to the 90 s

46、econd limitation (older models) on the battery backup, the write pending limits on the RMS is far less than 80% sometimes going as low as 12%. The limit is calculated at time of install.374 Layout for 1 DMX800 & 8 DAE40 U Cabinet1 U = 1.75 inches6 U (10.5 inches)1 U (1.75 inches)3 U (5.25 inches)1 U

47、 (1.75 inches) Each375DMX800 Rear view Here is an actual view of the rear of a DMX800, notice the cable management to enable cables to follow the SPE when it is pulled forward from the front of the unit.SPE with folding cable guide4 - DAEs3 - SPSs376SymmWin LauncherFor every SPE attached to the 1u s

48、erver, there will be an instance of SymmWin and SymmOptimizer. The 1u server can host up to 4 SPE units.If one of the SPEs had an error condition, the block would turn red. 377EnvironmentalsThe RMS has no ECM or CCM, the monitoring is a shared function.378Enginuity Code 5671Changes379TBC vs. LRULRU

49、machines must be converted to Tag Based Cache (TBC) before Non-Disruptive Upgrade (NDU)Enabling or Disabling TBC online:5670.69.64 and below: If the number of cache slots is greater than1,000,000 (0 xF4240; check with A1, command), then this setting can only be changed offline.5670.72.69 and above:

50、Proceed online, even if your cache exceeds 1,000,000 slots.5670 5671 Upgrade380Power Calculator 5670A minimum amount of power supplies must be provided to support the proposed configurationInput parameters used for the power calculation:Model of the SymmetrixNumber of Host Directors Number of physic

51、al Drives Type of physical Drive.Cache and Power Calculator 5671The Cache and Power Calculator in SymmWin will report:Minimal combination of memory boards legal for the configurationMinimum cache size which is legalNumber of power supply units required for the configurationCalculated power consumpti

52、on of the system 381Configuration Rules 5670Currently there are some soft limits in the configuration that the CS representatives must manually check for within every configuration.Maximum allowed Splits on a drive for each protection typeMaximum allowed Targets per DAMaximum allowed Customer Logica

53、l VolumesDMX800: 10K and 15K drives can NOT be mixed in the same DAE (primus solution emc81668)DMX10003000: Mirroring across 10K and 15K drives creates performance issues. Customers complain of inconsistent responses times382In 5x71 code SymmWin includes “Policing” rules that make sure all of the co

54、nfigurations are checked for all of the known rules, at every SymmWin “Save” (green check box or file save).The mirroring rules are the same as for 5670 code but now the SUPPLEMENTAL rules are also automatically assessed (Including Front End / Back End balancing)These rules, now built into SymmWin,

55、work for new installs and all upgrade actions.SymmWin Policing 5671383 SymmWin Policing 5671: Memory CheckRegardless of previous values/no values, loading the BIN-file to the Symmetrix.384 SymmWin Policing 5671: DA TargetsDA Targets are being reported by SymmWin as outside the allowed limitsNotice t

56、hat the event is an Error. A configuration in Error cannot be loaded on the Symmetrix.385 SymmWin Policing 5671: Splits per DriveSplits per drive are being reported by SymmWin as outside the allowed limitsNotice that the event is an Error. Note: A configuration in Error cannot be loaded on the Symme

57、trix.386Legal Board CombinationsCheck for legal combinations of:Front-end (host connection) boardsBack-end boardsMemory boardsWhen the user clicks the “apply” button in the “directors map” editor, SymmWin checks:Number of host boards or back-end boards is legalBoards are configured in legal slots567

58、0:SymmWin reports only on an illegal combination of emulations on the same board.If the memory boards information exists, the number of host boards is checked with regard to the number of memory boards. 387Logical Volumes 5671To grow with box capacity, code must support a larger number of logical de

59、vicesTherefore, 5671 extends the maximum number of customer-configurable logical devices to 16k (true 16,000 not 16*1,024)16,000 external volumes384 internal volumesReleasePlatformTotalExternalInternal5671Symm616,384(0 x4000)16,000384388Dynamic Spare vs. Global SpareDynamic or Hot spare is invoked a

60、gainst bad drive and takes up a mirror position.Data is copied to the Hot spare and bad drive is made NRWhen bad drive is replaced data is copied back to new driveHot spare is released and is ready to be used againGlobal or Permanent member spare is invoked and replaces bad drive. Data is copied to