ansibleszelechoski-ansible-3-phpapp02

1、How to Get More Sleepand Require Less CoffeeSarah Zelechoski szelechoski Providence VMUG 06/19/2014State of OpsWhy We Sleep on the CouchWere the ones they call when “The website is down” The customer is having an “weird” issue A critical exploit just entered the wild “I cant login to the domain” New

2、 deployment off hours Offshore needs 6 new VMs for testing “Im doing a customer demo at 8AM”Why We Own DNKNWere seriously overloaded Building VM templates Configuring clusters Storage management Networking Software deployment User/Group Authentication/Authorization Performance Testing OS Upgrades So

3、ftware Upgrades Security Patching Troubleshooting Customer Support Development Labs Alerting/MonitoringGuys What R U Doin Building VM templates ISO install and configuration Network setup Set up users/group, security, authentication/authorization Software install and configurationSoftware deployment

4、sTurn off monitoring/alertingPull nodes out of Load Balanced GroupRun DB migrations Deploy application code Restart web serverPut nodes back in/turn monitoring back on Building out clusters Cloning N number of VMs from X number of templates Hostname/network configuration FirewallingServer maintenanc

5、e SSH in to every server and restart a service Write complex scripts to log in to every server and update opensslGuys STAHP Ad hoc is bad hoc Complex shell scripts to account for every eventuality DRY (yeah I used to be a ruby dev) Any manual task can introduce human error They shouldnt have to call

6、 you on your on vacationPut an M on It Configuration Management (CM) authoritative centralization of configuration data and actions history of updates, changes for auditing purposes define the exact state a system should be in Infrastructure CM define the state that a system should be in with respec

7、t to its configuration and use tools that achieve that state enforce consistency across an entire environment automate to increase efficiency and repeatability easier to affect change (cloud provider, OS, etc.) remove the human factor disaster recoveryTool Time Puppet great with Windows (as long as

8、theyre not XP) amazing Enterprise support cryptic DSL (imo) Chef easy to learn if youre a ruby developer! amazing wealth of cookbooks Almost too verbose SaltStackAnsible Agentless! Uses SSH (with one python requirement) Everything is a YAML file Structure is flexible (ad-hoc, playbooks, roles, orche

9、stration) Easily extensible via modules Encryption and security built in Full power at the CLI (open source!) Even more features available in enterprise (Tower) No Windows IdempotentIdempo-what?“Operations in mathematics and computer science, that can be applied multiple times without changing the r

10、esult beyond the initial application.” wikipediaIdempodent: ExampleYou need all your application servers tomcat setenv.sh to look like:JAVA_HOME=/usr/java/latestJAVA_OPTS=-Xms512m -Xmx1024m -XX:PermSize=128m -XX:MaxPermSize=256m CATALINA_HOME=/usr/local/tomcatYou could get the job done with a classi

11、c echo commandecho JAVA_HOME=/usr/java/latest /usr/local/tomcat/bin/setenv.shecho JAVA_OPTS=-Xms512m -Xmx1024m -XX:PermSize=128m -XX:MaxPermSize=256m /usr/local/tomcat/bin/setenv.shecho CATALINA_HOME=/usr/local/tomcat /usr/local/tomcat/bin/setenv.shBut what if these lines already exist? what if the

12、file doesnt exist on a few of the servers? what if you needed to run your script again to update/restore another setting? If you dont know the beginning state of the system, the end state is unpredictableIdempotent: Example You could write a more complex scriptfor i in $tomcat_env_filedoecho Process

13、ing file $i# i hate this trick, but since a non match thatis status code 1 and# will kill this script do an unless here that forces a zerohas_java_home=$(grep -cE JAVA_HOME= $i | true)if sed felif$has_java_home -eq 0 ; thensetenv-without-java-home.sed -i $i $has_java_home -eq 1 ; thensed -f setenv-w

14、ith-java-home.sed-i $ielseecho Something went very wrong.Please review $i and makesure there is only a single line containing JAVA_HOME= and runscript againfi doneAint Nobody Got Time For That Use an idempotent CM tool Tell the tool what state you need the system to be in Tool will get you from A-Z

15、or B-Z or even C-Z Wont get mangled configs Wont get conflicting packages Wont get mismatched versions Wont get error messages you have to handle for each unique caseAnsible: Example template the setenv.sh file we want# ansible_managed JAVA_HOME=java_homelatest JAVA_OPTS=-Xms512m -Xmx1024mCATALINA_H

16、OME=tomcat_home-XX:PermSize=128m-XX:MaxPermSize=256m provide defaults (in yml!)-java_home: /usr/java/ tomcat_home: /usr/local/tomcattomcat_user: name: tomcat, group: tomcatAnsible: Example write ansible task- name: Apache Tomcat | Configure | Overlay configurationtemplate: src=setenv.j2 dest=tomcat_

17、home/bin/setenv.sh owner=tomcat_ group=tomcat_user.group profit!$ cat /usr/local/tomcat/bin/setenv.sh# Ansible managed: /tmp/packer-provisioner-ansible-local/roles/roles/tomcat- 7/templates/cloud/app/setenv.j2 modified on 2014-04-04 13:18:35 by vagrant vagrantJAVA_HOME=/usr/java/latestJAVA_

18、OPTS=-Xms512m -Xmx1024m -XX:PermSize=128m -XX:MaxPermSize=256m CATALINA_HOME=/usr/local/tomcatonStructure Inventory Modules Ad Hoc Commands Playbooks Tasks Variables Templates Handlers RolesInventory: Exampleproduction:children webserversdbservers http_port=80 bar.ex

19、 http_port=8080dbserversdb01:03.dbservers:varspgsql_bind_nic=eth1proxies$ ansible production a “echo hello” u joek$ ansible dbservers a “service postgresl restart” u joe U root k -KDynamic vSphere Inventory Any script that can output JSON can be used to generate dynami

20、c inventory Use pysphere (python) or rbvmomi (ruby) to communicate with vSphere/vCenter Organize your VMs by folder or resource pool to translate in to groupDynamic vSphere Inventory /RaymiiOrg/ansible-vmware$python2 query.py -listno_group: hosts: vm0031,vm0032, . vm0999,local: 127.

21、0.0.1Dynamic vSphere InventoryPreface each command with the script$ANSIBLE_HOSTS=/src/vmware-ansible/query.py ansibleall -m pingExport an environmental variable$export ANSIBLE_HOSTS=/src/vmware-ansible/query.py”ansible no_group -m pingModules can be written in any language as long as they output JSO

22、N take parameters and conditions to define desired state handles processing of system resources, services, packages, files, etc. in idempotent fashion “seek to avoid changes to the system unless a change needs to be made” ansible comes preloaded with a plethora of modules tons of community pull requ

23、estsAd Hoc Commands run a single, one-off command run on a full or partial inventory run on a single host no need to save for later$ ansible webservers m command a “dpkg-querySSH password: | success | rc=0 openssl 1.0.1e-2+deb7u10W openssl” u joe | success | rc=0 open

24、ssl 1.0.1e-2+deb7u10Playbooks More powerful configuration management Kept in source control, developed, validated Declare configurations of more complex mutli-system enviornments Arrange and run tasks synchronously or asynchronouslyPlaybooks: Example- roles/ansible- hosts: all remote_user: vagrant s

25、udo: true sudo_user: root vars_files:- roles/vars/webserver.encrypt vars:lifecycle: dev roles:- roles/debian- roles/vmware-tools- roles/local-users- roles/sudoers- roles/iptables- roles/clamav- roles/java-jdk-7- roles/postgres- roles/apache- roles/tomcat-7- role: roles/tc-native, when: native= true

26、- roles/git- roles/liquibase- roles/cleanup post_tasks:- name: Reboot the Server command: /sbin/reboot- name: Wait for Server to come backwait_for: host=inventory_hostname port=22 sudo: nodelegate_to: localhost- name: Wait for Services to start fullywait_for: port=item delay=5 timeout=600 with_items

27、:- 8009 #ajp- 8080 #tomcat- 80 #httpdPlaybooks: Example$ansible-playbookiproductionwebserver.ymlk K$ansible-playbookiproductionwebserver.ymlf 10 k K$ansible-playbookiproductionwebserver.yml-list-hosts -k K$ansible-playbookiproductionwebserver.yml-check k KTasks: Examplemoduleparameteriteratorvariabl

28、e-name: Apache Tomcat | Install | Grab latest tomcat tarballget_url: url=tomcat.base_urlitem.sub_urlitem.file dest=/tmp/item.file with_items: tomcat.files-name: Apache Tomcat | Install | Extract archiveshell: tar -xvzf /tmp/item.file -C /usr/local creates=/usr/local/item.target with_items: tomcat.fi

29、les-name: Apache Tomcat | Install | Give ownership of install to tomcat userfile: path=/usr/local/item.target state=directory owner= group=tomcat.user.group with_items: tomcat.files-name: Apache Tomcat | Install | Symlink install directoryfile: src=/usr/local/item.target path=/usr/lo

30、cal/tomcat state=link with_items: tomcat.files-name: Apache Tomcat | Configure | Overlay configurationtemplate: src=item.file dest=item.target owner= group=tomcat.user.groupwith_items: tomcat.config_filesVariables: Simple YAML format Can create arrays and hashes Can substitute vars i

31、nto vars Vars can be defined at many levels (default, role ,playbook) Can test conditionals on vars and require them Can be filtered and manipulated with jinja2 Can be matched to regex!Variables: Example# # TOMCAT #tomcat_home: /usr/local/tomcat tomcat:config_files:-file:file:file:tomcat.j2,setenv.j

32、2,server.j2,target:target: target:/etc/init.d/tomcattomcat_home/bin/setenv.sh tomcat_home/conf/server.xmldefault_webapps: ROOT,docs,examples,host-manager,managerbase_url: /pub/apache/tomcat/tomcat-7/v7.0.54/bin/files:- file: apache-tomcat-7.0.54.tar.gz, target: apache-tomca

33、t-7.0.54 user: name: tomcat, group: tomcatTemplates Templates are interpreted by jinja2 stub out files fill variables in differently depending on conditions Powerful conditionals Loops and iterators Replace a file completely every time? Yes. We configure for an end state.Templates: Example# ansible_

34、managed DefaultsDefaults Defaultsenv_resetmail_badpasssecure_path=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin”# User privilege specificationroot# Allow%sudoALL=(ALL:ALL)members of group ALL=(ALL:ALL)ALLsudo to execute ALLanycommand% for item in admins % if item.nopasswd is true %ite

35、ALL= NOPASSWD:ALLDefaults: !requiretty% else %ALL=(ALL)ALL%endif %endfor %if ad is defined %for item in ad.sudoers_groups %itemALL=(ALL)% endfor % endif %ALLTemplates: Example# Ansibledebian.j2 Defaults DefaultsDefaultsmanaged: /tmp/packer-provisioner-ansible-local/roles/role

36、s/sudoers/templates/sudoers-modified on 2014-06-09 10:08:44 by vagrant on vagrant env_resetmail_badpasssecure_path=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin”# User privilege specificationroot# Allow%sudoALL=(ALL:ALL)members of group ALL=(ALL:ALL)ALLsudo to execute any command ALLy

37、odaALL= NOPASSWD: ALLDefaults:yoda !requirettylukeanakinALL=(ALL)ALL=(ALL)ALLALL%jediALL=(ALL)ALLHandlers Written just like a regular task Only run if triggered by the notify directive Indicates a change in the system state Any module can be used for the handler action Handler- name: Restart Tomcats

38、ervice: name=tomcat state=restartedTask- name: Apache Tomcat | Configure | Overlay configuration template: src=item.file dest=item.target with_items: tomcat.config_filesnotify: Restart TomcatRoles Break up configuration into repeatable chunks Reduce, reuse, recycle Clean, understandable structure St

39、ack on top of each other Ansible GalaxyRoles active directory ansible apache app-dynamics artifactory centos clamav cleanup debian git gradle iptables java-jdk-6 java-jdk-7 jenkins keystore liquibase local-users postgres sudoers swarm packer tomcat-6 tomcat-7 tomcat-native vmware-tools zeromqRoles S

40、tructureroles/tomcat-7/defaults/main.yml files/blah.txt handlers/main.yml meta/main.ymltasks/main.yml templates/setenv.j2 tomcat.j2 server.j2 limits.j2vars/main.ymlRoles Dependencies Always run before the roles that depend on them If dependencies are duplicated amongst roles, they will only be run o

41、nce by default Can use allow_duplicates to require a role to be run more than once with different conditions-dependencies:-role:role: role:liquibase apache, port: 80 postgres, dbname: appdb, bind_nice: eth1 Orchestration “Rolling Updates” Performing very complex infrastructure or cluster operations

42、Run plays in serial instead of parallel Wait for certain conditions to move forward Abort if certain percentage of failureOrchestration: Example turn off monitoring and alerting remove application server from load balanced group stop services wait for services to stop fully checkout new code from gi

43、t deploy webapp restart services wait for services to start fully return to load blanced groupExample: Simple Service Restart Problem 50ish production customer VMs Older CentOS 5 mixed with CentOS 6 May or may not have python installed Domain authentication Need to restart livevault serviceExample:

44、Simple Service Restart Create inventory Dump IP addresses of customer VM into simple ansible inventorycustomer_vms:varscustomer_vms17976800978158.Example: Simple Service Restart Use an ad hoc command to m

45、ake sure VMs are bootstrapped for Ansible$ ansible cusomter_vms -i oldvms u domainjoe -s -U root-a sudo yum install -y python-simplejson -k K-m raw Restart the live vault service$ ansible customer_vms i oldvms u domainjoe s U rootservice a name=livevault state=restarted k -K-mExample: Heartbleed ope

46、nssl exploit good news: patched for your OS other packages updated along with openssl 6 different environments (production, test, demo, etc.) may require service restarts need verification of final state versionExample: Heartbleed-hosts: allsudo: yes sudo_user: root tasks:-name: OpenSSL | Get curren

47、t versionshell: dpkg-query -W openssl register: openssl_version-name: OpenSSL | Get current versionshell: dpkg-query -W libssl1.0.0register: libssl_version-name: OpenSSL | Confirm new versiondebug: msg=OpenSSLinstalled isversion installed is openssl_version.stdout,libssl_version.stdoutlibssl version

48、-name: OpenSSL | Apt| Install debconf-utilsapt: pkg=debconf-utils state=latestExample: Heartbleed-name: OpenSSL | Apt | Prevent restart services dialogdebconf: name=libssl1.0.0 question=libssl1.0.0/restart-services vtype=string value=ntp-name: OpenSSL | Apt | Prevent restart services dialogdebconf:

49、name=libssl1.0.0:amd64 question=libssl1.0.0/restart-services vtype=stringvalue=ntp-name: OpenSSL | Aptapt: pkg=item with_items:- openssl- libssl1.0.0| Upgrade Opensslstate=latestupdate_cache=yesinstall_recommends=yesforce=yes-name: OpenSSL | Getnew versionshell: dpkg-query -W opensslregister: openss

50、l_version-name: OpenSSL | Get new versionshell: dpkg-query -W libssl1.0.0 register: libssl_version-name: OpenSSL | Confirm new versiondebug: msg=OpenSSL version installed is openssl_version.stdout, libssl version installedlibssl_version.stdoutisExample: Heartbleed$ ansible-playbook -i cloud-daily, openssl.yml -u joe -k -K SSH password:sudo password defaults to SSH password:PLAY all *GATHERING FACTS *ok: cloud-dailyTASK: OpenSSL | Get currentchanged: cloud-dailyversion*TASK: OpenSSL | Get currentchanged: cloud-dail