Download Orchestration.pdf
Read online
The author approaches orchestration from the premise that the principles can best be presented by analysis of musicas it has been written.The essentials of (Orchestration )] [Author: Walter Piston] [Apr-1955] [Walter Piston] Orchestration [Walter Piston] In this book Walter Piston again displays those qualities that distinguished Books on Orchestration, Film Scoring, and Music Theory. Below, I've Orchestration by Walter Piston The Study of Orchestration by Samuel Adler. I'm listing Again he draws upon his own wide knowledge and experience as composer and teacher to present all phases of the subject. No practical aspect of Cohen Music Cognition and the Cognitive Psychology of Film Structure. Cohen (1998) The function of music in multimedia. Paul Bocuse Les Meilleures Books by Walter Piston. HARMONY COUNTERPOINT. ORCHESTRATION. by. WALTER PISTON. PROFESSOR OF MUSIC, HARVARD UNIVERSITY. LONDON. Sheet Music - £39.75 - Piston's definitive book on Orchestration proves as popular as ever and is considered to be the bible for anyone, professional or Walter Piston home page at www.boosey.com. included Harmony (1941), Counterpoint (1947) and Orchestration (1955) Works by Walter Piston include: Again he draws upon his own wide knowledge and experience as composer and teacher to present all phases of the subject. No practical aspect of
1 Protocols: DNS, TELNET, e-Mail, FTP, WWW, NNTP, SNMP, NTP etc.
Lecture #21: Application LayerProtocols: DNS, TELNET, , FTP, WWW, NNTP, SNMP, NTP etc.
2 TCP/IP Application Layer
User application 1...Encryption/decryptioncompression/expansionChoice ofsyntaxPresentation layerSessioncontrolSessionsynch.Session to transportmappingSessionmanagementSession layerLayer and flowcontrolErrorrecoveryMultiplexingTransport layerConnection controlRoutingAddressingNetwork layerData link establishmentErrorcontrolFlowcontrolSynchFramingLink layerAccess to transm. mediaPhysical and electrical interfaceActivation/ deactivation of con.Physical layerOSI layersWide used Application protocols: DNS, HTTP, FTP, SMTP, POP3, IMAP, NNTP, SNMP, NTP, RPC, TELNET, SSH ...
An essential update to the key web authoring standards of HTML, XHTML, and CSS The existence of Web pages depends on three vital technologies: HTML (base language that Web pages are written in), XHTML (standards that define how to write HTML pages), and CSS (standard that applies formatting styles to Web pages). Dec 28, 2007. The CGI looks horrendous; not so much the dynamic stuff (the Ents look fine) but the super-fake sets. Be between Terrorism and Security', and fretted that to live by the security experts' worst-case scenarios might be to surrender too many of out liberties to the invisible shadow warriors of the secret world.
3 DNS - The purpose To map a hostname to an IP-address and vice versa.
Example:Each hostname may has zero or more corresponding IP-addresses.Each IP-address may correspond to zero or more hostnames.
4 DNS - Historical remarks
Host name to address mappings were maintained by the Network Information Center (NIC) in a single file (HOSTS.TXT) which was transferred to all hosts using FTP (about 25 years ago).Disadvantages:Ineffective – Bandwidth consumed in distributing a new version by this scheme is proportional to the square of the number of hosts in the network.The network population was also changing in character.The applications on the Internet were getting more sophisticated and creating a need for general purpose name service.
5 DNS design goalsThe primary goal is a consistent name space which will be used for referring to resources.The database which holds the names must be distributed.The costs of implementing such a facility dictate that it be generally useful, and not restricted to a single application.
6 DNS design goals (contd.)
Use the same name space with different protocol families or management.Name server transactions must be independent of the communications system that carries them.The system should be useful across a wide spectrum of host capabilities. Both personal computers and large timeshared hosts (mainframes) should be able to use the system, though perhaps in different ways.
7 Elements of the DNS Three main componets
The DOMAIN NAME SPACE and RESOURCE RECORDS, which are specifications for a tree structured name space and data associated with the names.NAME SERVERS are server programs which hold information about the domain tree's structure and set information.RESOLVERS are programs that extract information from name servers in response to client requests.
8 Elements of the DNS (contd.)
These three components roughly correspond to the three layers or views of the DNS:From the user's point of view, the domain system is accessed through a simple procedure.From the resolver's point of view, the domain system is composed of an unknown number of name servers.From a name server's point of view, the domain system consists of separate sets of local information called zones.
9 Name space specifications and terminology
The domain name space is a tree structure composed by nodes.Each node has a label, which is zero to 63 octets in length.The domain name of a node is the list of the labels on the path from the node to the root of the tree.The name of the root node is empty string (zero length) – “”.
10 Name space specifications and terminology (contd.)
Internally, programs that manipulate domain names should represent them as sequences of labels, where each label is a length octet followed by an octet string.Domain names are case-insensitive using the ASCII codes with high order zero bit.When a user types a domain name, the length of each label is omitted and the labels are separated by dots ('.'). Since a complete domain name ends with the root label, this leads to a printed form which ends in a dot.
11 Example Name Space (a graphical tree view)
ICANN, IANAOne Root domain (“”)Top Level Domains (TLD)Delegationcomedubgeuinfoyahoouclaacadsee-grid2nd Level Domainsphysicsamigovalwww3rd Level DomainsmaryjoekevintestbookmailsigmaUp to 127 levels
12 Top Level Domains Generic domains (gTLD):
com, org, net, int (all around the world);edu, gov, mil (located in USA only);aero, biz, coop, info, museum, name, pro(new domains, since 2000).Country code domains (ccTLD):bg, uk, fr, it, ...Special infrastructure domain:arpaLoopback domains:localhost, localdomainAnd the domain eu for European Union!
13 Resource RecordsThe distributed database which holds the names consists of several Resource Records.Resource record - Set of resource information associated with a particular name.
14 DNS server A computer with software which:
Holds and maintains specified part of the distributed database of resource records.Responds to the name queries.Exchanges the zone information with other DNS servers.These tasks are described in the DNS protocol (RFC 1035) located on the Application Layer of the “TCP/IP reference model”.
15 The root DNS serversAt the heart of the DNS are 13 special computers, called root servers. They are coordinated by ICANN and are distributed around the world.A.ROOT-SERVERS.NETB.ROOT-SERVERS.NETC.ROOT-SERVERS.NETD.ROOT-SERVERS.NETE.ROOT-SERVERS.NETF.ROOT-SERVERS.NETG.ROOT-SERVERS.NETH.ROOT-SERVERS.NETI.ROOT-SERVERS.NETJ.ROOT-SERVERS.NETK.ROOT-SERVERS.NETL.ROOT-SERVERS.NETM.ROOT-SERVERS.NET
16 K.ROOT-SERVERS.NET (operated by RIPE NCC)
Source:
17 DNS servers (contd.) Delegation:
ICANN organization delegates root DNS servers.The root servers delegate TLD DNS servers.A TLD DNS server delegates Second Level DNS server.etc.Primary DNS server for a given domain – holds resource records for the zone. The zone configuration is written by the system administrator.Secondary DNS server for a given domain – it exists for the case when the Primary DNS server is failed or overloaded. The Secondary server downloads the zone information from the Primary periodically or by notification.
18 DNS servers (contd.)Every domain must have one Primary DNS server and at least one Secondary.The Primary and Secondary DNS servers are also called authoritative for their domain and non-authoritative for the rest of domains in the world. E.g. the authoritative servers always hold up to date zone information for the domain.Caching only DNS servers – they cache DNS queries and answers for speed-up the service. These servers are not authoritative for any domain.Most of the authoritative DNS servers are caching too.
19 DNS query flowA DNS client forms the query and sends it according the OS configuration.The local resolver opens a special file called hosts and looks in case the answer is already written here. If not then next:The client calls one of the OS configured DNS servers.If the DNS server knows (or already cached) the answer, then returns it.If the DNS server doesn’t know the answer, it sends a recursive query through the DNS hierarchy, starting from the root domain. The recursive query consists of several sub-queries to the corresponding DNS servers.The DNS servers returns the result – the answer or error message.
20 Reverse DNS queryThe reverse query uses the special infrastructure domain in-addr.arpaThe IP is written before this special domain in reverse order of numbers separated with dots.Example: corresponds to in-addr.arpaThe reverse query starts from a root DNS server and follows the delegation path.Example in Unix/Linux:dig in-addr.arpa +trace(It traces and shows the delegation path from the root name servers down to SOA DNS server)
21 DNS protocol detailsThe protocol uses two transport protocols from the lower layer:UDP on port 53 andTCP on port 53.The queries and answers are transferred using the unreliable protocol UDP for speed. But it is preferred because the OS configured DNS servers are near, usually in the same LAN.The zone transfers between the DNS servers are performed through TCP, because it must be reliable.
22 DNS protocol details (contd.) Resource Records described
Each resource record consists of the following data fields( in text readable format! ):owner type class TTL RDATA
23 Resource Records The data fields
owner - A string which is the domain name where the RR is found.type - which is an encoded 16 bit value that specifies the type of the resource in this resource record. Types refer to abstract resources. Each abstract resource has memo name:A - a host addressCNAME - identifies the canonical name of an aliasMX - identifies a mail exchange for the domainPTR - a pointer to another part of the domain name spaceetc.
24 DNS protocol details (contd.) Resource Records The data fields
class - which is an encoded 16 bit value which identifies a protocol family or instance of a protocol.This memo uses the following classes:IN - the Internet system or CH - the Chaos systemTTL - which is the time to live of the RR. This field is a 32 bit integer in units of seconds, an is primarily used by resolvers when they cache RRs. The TTL describes how long a RR can be cached before it should be discarded.RDATA - which is the type and sometimes class dependent data which describes the resource.
25 Resource Records Example
(in text readable format!)acad.bg IN Aacad.bg IN MX mx-a.acad.bg.IN MX mx-b.acad.bg.IN NS amigo.acad.bg.IN NS unicom.acad.bg.localhost IN Aposeidon IN Abackbone IN Airis IN Aamigo IN AAAA :4b58:acad::3unicom IN AAAA :4b58:acad::9bis IN CNAME unicom.acad.bg.IN PTR amigo.acad.bg....
26 DNS protocol details (contd
DNS protocol details (contd.) The DNS message in binary format (RR means Resource Record)
27 DNS protocol details (contd.) The DNS message data fields
The IDENTIFICATION field is set by the client and returned by the server.The 16-bit PARAMETER consists of:0-th bit field: 0 means the message is a query,1 means it's a response.1-4 bit fields - OPCODE:is a normal value (Standard query).an inverse query.the server status request.5-th bit field - Authoritative answer. The name server is authoritative for the domain in the question section.6-th bit field is set if message truncated. With UDP this means that the total size of the reply exceeded 512 bytes, and only the first 512 bytes the of the reply were returned.7-th bit field - Recursion Desired. This bit can be set in a query and is then returned in the response.8-th bit field - Recursion Available.9-11 -th bits field has to be 0.th bits field - Return Code. 0- no error, 3- name error.
28 DNS protocol details (contd.) The DNS message data fields
The fields labeled NUMBER OF ... give each a count of entries in the corresponding sections in the message.The QUESTION SECTION contains queries for which answers are desired. The client fills in only the question section; the server returns the question and answers with its response. Each question has Query Domain Name followed by Query Type and Query Class fields (as depicted in the next slides)ANSWER,AUTHORITY,ADDITIONAL INFORMATION sections consist of a set of resource records that describe domain names and mappings. Each resource record describes one name (as depicted in the next slides)
29 DNS protocol details (contd
DNS protocol details (contd.) The QUESTION section format in the DNS messageThe DOMAIN NAME has variable length.Clients fill in the questions; servers return them along with the answers.
30 DNS protocol details (contd
DNS protocol details (contd.) The RESOURCE RECORD binary format returned by the DNS servers
31 DNS Further readingInterNIC organization - provides the public information regarding Internet Domain Name registration services. -ICANN - responsible for the global coordination of the Internet's system of unique identifiers -RFC1034, RFC1035, STDBIND (Berkeley Internet Name Domain) – the most popular implementation of the DNS software -Paul Albitz, Cricket Liu, “DNS and BIND”, 4th edition, O’REILLY
32 TELNET protocolTELNET (TELecommunication NETwork) was developed in 1969 beginning with RFC 15 and standardized as IETF STD 8, one of the first Internet standards.TELNET clients have been available on most Unix systems for many years and are available for virtually all platforms. Most network equipment and OSs with a TCP/IP stack support some kind of TELNET service server for their remote configuration (including ones based on MS Windows NT and later).Because of security issues with TELNET, it use has waned as it is replaced by the use of SSH for remote access.
33 TELNET protocol (2)Most often, a user will be telneting to a Unix-like server system or a simple network device such as a switch. Once the connection is established, he would then log in with his account information and execute operating system commands remotely on that computer, such as ls or cd etc.For testing and debugging purposes: On many systems, the client may also be used to make interactive raw-TCP sessions, even when that option is not available. The sessions are equivalent to raw TCP as long as byte 255 never appears in the data.TELNET works on the well known TCP port 23.
34 Electronic mailBefore 1990, it was mostly used in academic area - universities and research centers.During the 1990s, it became known to the public at large and grew exponentially.The first systems simply consisted of text file transfer protocols. As time went on, the limitations of this approach became more obvious.Some of the complaints were as follows:Sending a message to a group of people was inconvenient.Messages had no defined internal structure.The sender never knew if a message arrived or not.Lack of 'I'm temporary away' management.Poor user interface.It was not possible to create and send messages containing a mixture of text and binary data: drawings, photos, facsimile, and voice.
35 E-mail: Architectures and services
Typically, systems support five basic functions:Composition refers to the process of creating messages and answers.Transfer refers to moving messages from the originator to the recipient. The system should do this automatically, without bothering the user.Reporting has to do with telling the originator what happened to the message.Displaying incoming messages is needed so people can read their . Sometimes conversion is required or a special viewer must be invoked.Disposition is the final step and concerns what the recipient does with the message after receiving it. (Delete, Replay, Forward, Save ...)
36 Electronic Mail The User Agent
Paper mail
37 ReadingAn example of mailbox
38 Message Formats – RFC 822
39 MIME – Multipurpose Internet Mail Extensions
The original was designed to transfer 7-bit text (ASCII) characters only, so ...Problems with international languages:Languages with accents (French, German).Languages in non-Latin alphabets (Hebrew, Cyrillic ...).Languages without alphabets (Chinese, Japanese).Messages not containing text at all (audio or images).
40 MIME
41 MIME – a multipart message example
42 Message Transfer example using SMTP
Transferring a message from to
43 Final deliverySending and reading mail when the receiver has a permanent Internet connection and the user agent runs on the same machine as the message transfer agent.Reading when the receiver has a dial-up connection to an ISP.
44 Post Office Protocol (POP3)
Using POP3 to fetch three messages.
45 Internet Message Access Protocol (IMAP. TCP port 143)
A comparison of POP3 and IMAP.The current version of IMAP is 4 revision 1 – RFC 3501
46 File Transfer Protocol (FTP)
File Transfer Protocol (FTP) is the standard mechanism provided by TCP/IP for copying of any kind of files from one host to another.Defined in RFC 959 (1985).FTP uses the services of TCP. It needs two TCP connections.The well-known TCP port 21 is used for the control connection and the well-known port 20 for the data connection.
47 FTP operationFile transfer
48 FTP – the file transfer proccess
49 FTP – an example session
50 FTP - access commands
51 FTP – file management commands
52 FTP – data formatting commands Port definition commands
53 FTP – file transfer commands
54 FTP – Miscellaneous commands
55 FTP – Some responses
56 Trivial FTP (TFTP)A very simple file transfer protocol, with the functionality of a very basic form of FTP. Defined in 1980.TFTP uses UDP port 69 (not TCP!)An example of TFTP usage for remote boot through BOOTP protocol in a LAN:
57 FTP pros and cons Advantages: Disadvantages: Simple implementation
Universal applicationWide used and standartizedDisadvantages:Clear text passwords, unencrypted data.Multiple TCP/IP connections are used = > Firewalls problems.Hard to filter active mode FTP traffic on the client side by using a firewall.It is possible to abuse the protocol's built-in proxy features to tell a server to send data to an arbitrary port of a third computer; see FXP.High latency.No integrity check on the receiver side.No date/timestamp attribute transfer.
58 World Wide Web Architectural Overview
The parts of the Web model.
59 The Client Side(a) A browser plug-in (b) A helper application.
60 The Server SideA multithreaded Web server with a front end and processing modules.
61 The Server Side (2)A server farm.
62 The Server Side (3) (a) Normal request-reply message sequence.
(b) Sequence when TCP handoff is used.
63 The World Wide Web - HTTP
Hypertext Transfer Protocol (HTTP) is a communications protocol used to transfer or convey information on intranets and the World Wide Web. Its original purpose was to provide a way to publish and retrieve hypertext pages, mainly scientific paper. Created in CERN, Geneva.Development of HTTP was coordinated by the W3C (World Wide Web Consortium) and the IETF (Internet Engineering Task Force), culminating in the publication of a series of RFCs, most notably RFC 2616 (June 1999), which defines HTTP/1.1, the current version.HTTP is a request/response protocol between a client and a server. It works on the well known TCP port 80.
64 HTTP - methodsHTTP - responses
65 HTTP - Some Message Headers
66 test.log partial content
Example HTTP usage in a TELNET session: telnet 80 >test.log GET /rfc.html Connection closed by foreign host.test.log partial contentTrying 2610:a0:c779:b::d1ad:35b4...Connected toEscape character is '^]'.<HTML><head><TITLE>IETF RFC Page</TITLE><SCRIPT LANGUAGE='JavaScript'>function url() {var x = document.form1.number.valueif (x.length 1) {x = '000' + x }if (x.length 2) {x = '00' + x }if (x.length 3) {x = '0' + x }document.form1.action = ' + x + '.txt'document.form1.submit}</SCRIPT></head><!-- begin new headers and page layout --><body text='#000000' bgcolor='#ffffff' ><center>...
67 Hierarchical caching with 3 proxies.
HTTP - CachingHierarchical caching with 3 proxies.
68 URLs – Uniform Resource Locaters
Some common URLs.
69 Statelessness and Cookies
Some examples:
70 HTML – HyperText Markup Language
In 1980, physicist Tim Berners-Lee, who was an independent contractor at CERN, proposed and prototyped ENQUIRE, a hypertext system for CERN researchers to use to share documents. In 1989, Berners-Lee and CERN data systems engineer Robert Cailliau each submitted separate proposals for an Internet-based hypertext system providing similar functionality. In 1990, they collaborated on a joint proposal, the World Wide Web (W3) project, which was accepted by CERN.
71 HTML (2) July, 1993: a draft by IETF (that is: not a standard – yet).
November, 1995: HTML 2.0 published as IETF RFC 1866, supplemented by RFC 1867RFC 1942 (tables) in May 1996,RFC 1980 (client-side image maps) in August 1996, andRFC 2070 (internationalization) in January 1997;An HTML 3.0 standard was proposed in April 1995.January 14, 1997: HTML 3.2, published as a W3C Recommendation.HTML 3.2 was never submitted to the IETFDecember 18, 1997: HTML 4.0, published as a W3C Recommendation. It offers three 'flavors':Strict, in which deprecated elements are forbidden,Transitional, in which deprecated elements are allowed,Frameset, in which mostly only frame related elements are allowed;April 24, 1998: HTML 4.0 was reissued with minor edits without incrementing the version number.December 24, 1999: HTML 4.01, published as a W3C Recommendation. It offers the same three flavors as HTML 4.0, and its last errata was published May 12, 2001.HTML 4.01 and ISO/IEC 15445:2000 are the most recent and final versions of HTML.May 15, 2000: ISO/IEC 15445:2000 ('ISO HTML', based on HTML 4.01 Strict), published as an ISO/IEC international standard.HTML 5 is still an Editor’s Draft, and not endorsed by W3C yet.
72 HTML (3)(a) The HTML for a sample page. (b) The formatted page.(b)
73 HTML (3)A selection of common HTML tags. some can have additional parameters (attributes).
74 HTML Tables(a) An HTML table.(b) A possible rendition of this table.
75 HTML Forms(a) The HTML for an order form.(b) The formatted page.
76 HTML Forms (2)A possible response from the browser to the server with information filled in by the user.
77 XML and XSL eXtensible Markup Language
A simple Web page in XML.
78 XML and XSL eXtensible Stylesheet Language
A style sheet in XSL.
79 Dynamic Web DocumentsSteps in processing the information from an HTML form.
80 Dynamic Web Documents (2)
A sample HTML page with embedded PHP script.
81 “Web Services”The W3C defines a Web Service as “A software system designed to support interoperable Machine to Machine interaction over a network.“ Web services are frequently just Web APIs that can be accessed over a network, such as the Internet, and executed on a remote system hosting the requested services.The W3C Web Service definition encompasses many different systems, but in common usage the term refers to clients and servers that communicate using XML messages that follow the SOAP (Simple Object Access Protocol) standard. Common in both the field and the terminology is the assumption that there is also a machine readable description of the operations supported by the server written in the Web Services Description Language (WSDL). The latter is not a requirement of a SOAP endpoint, but it is a prerequisite for automated client-side code generation in the mainstream Java, .NET SOAP etc. frameworks.
82 Network News Transfer Protocol (NNTP)
An Internet application protocol used primarily for reading and posting Usenet articles (aka netnews or simply: news), as well as transferring news among news servers.Created by Brian Kantor of the University of California, San Diego.Phil Lapsley of the University of California, Berkeley completed RFC 977 (1986).Originally designed around the UUCP (Unix to Unix CoPy) network, with most article transfers taking place over direct computer-to-computer telephone links.
83 NNTP (2)Because networked Internet-compatible filesystems were not yet widely available, it was decided to develop a new text protocol that resembled SMTP, but was tailored for reading newsgroups.The well-known TCP port 119 is reserved for NNTP. TCP port 563 is used for connectiong through SSL (a.k.a. NNTPS).IETF released RFC 3977 in Oct 2006, which updates the NNTP protocol.The IMAP protocol can also be used for reading newsgroups.
84 An e-mail client as a News reader
Public News servers:
85 Simple Network Management Protocol (SNMP)
Application layer protocol within the OSI model. It uses UDP as transport.“The protocol that specifies how a network management station communicates with agent software in remote devices such as routers. SNMP defines the format of messages and their meaning.”- Computer Networks and Internets by Douglas E. Comer.
86 SNMP – some historySize and number of networks required a standard protocol in order to communicate with devices on the network.1970’s - SNMPv1, the first network management protocol.Originally designed as a “quick-fix” .SNMPv2 standard designed.2004 – SNMPv3, the current versionRFC 3411 – RFC 3418 (also known as STD0062).
87 SNMP Technology Exchanges network information through PDU’s.
Part of the Internet Network Management ArchitectureInternet Management ModelNetwork elementsAgentsManaged objectsMIB’s (Management Information Bases)Syntax notationSMI (Structure of Management Info)NMS (Network-management systems)PartiesManagement protocolsAgentMIBNetworkManagementApplicationUserInterfaceSNMPManaged DeviceNMS
88 SNMP OperationsSNMP is a simple “request/response” protocol which uses six operations:GetGetNextGetBulkSetTrapInform
89 SNMP v1 Packet Format SNMP v1 packets contain two parts:
First part contains version and community name.Second part contains PDU.VersionCommunitySNMP PDUPDUTypeRequestIDErrorStatusErrorIndexObject 1,Value 1Object 2,Value 2Object X,Value X ...
90 SNMP v2 Packet Format Like SNMP v1, SNMP v2 contain two parts:
First part is called a wrapper which contains authentication, privacy information and a context.Second contains a PDU with similarities to SNMP v1.WrapperSNMP PDUPDUTypeRequestIDErrorStatusErrorIndexObject 1,Value 1Object X,Value X ...OROR
91 SNMP v1 and v2 Advantages and Disadvantages
Security holes.Old technology.AdvantagesSimple design.Easy implementationWide spread usage.Expandability.
92 The new SNMP v3 New features include:
Structure, components and architecture similar to v1 and v2.New features include:Authentication and privacy.Authorization and access control.Naming of entities.People and policies.Usernames and key management.Notification destinations.Remotely configurable via SNMP operations.
93 SNMP ConclusionThe goals: Integrated network management, Interoperability, Standards.Network management applications based on SNMP rely on the standards based TCP/IP protocol to effectively oversee large heterogeneous networks.SNMP sits on the application layer and uses UDP protocol to communicate with each network device.SNMP is essential in order to effectively maintain today’s large networks.
94 NTP - IntroductionNetwork Time Protocol (NTP) synchronizes clocks of hosts and routers in the Internet.NTP provides nominal accuracies of low tens of milliseconds on WANs, submilliseconds on LANs, and submicroseconds using a precision time source such as a cesium oscillator or GPS receiver.NTP software has been ported to almost every workstation and server platform available today - from PCs to Crays - Unix, Windows, VMS and embedded systems, even home routers and battery backup systems.The NTP architecture, protocol and algorithms have been evolved over the last two decades to the latest NTP Version 4 software distributions.
95 Needs for precision time
Distributed database transaction journalling and loggingStock market buy and sell ordersSecure document timestamps (with cryptographic certification)Aviation traffic control and position reportingRadio and TV programming launch and monitoringIntruder detection, location and reportingMultimedia synchronization for real-time teleconferencingInteractive simulation event synchronization and orderingNetwork monitoring, measurement and controlEarly detection of failing network infrastructure devices and air conditioning equipmentDifferentiated services traffic engineeringDistributed network gaming and training... and Grid computing
96 NTP summaryPrimary (stratum 1) servers synchronize to national time standards via radio, satellite and modem.Secondary (stratum 2, ...) servers and clients synchronize to primary servers via hierarchical subnet.Clients and servers operate in master/slave, symmetric and multicast modes with or without cryptographic authentication.Reliability assured by redundant servers and diverse network paths.Engineered algorithms reduce jitter, mitigate multiple sources and avoid improperly operating servers.The system clock is disciplined in time and frequency using an adaptive algorithm responsive to network time jitter and clock oscillator frequency wander.
97 NTP architecture overview
Peer 1Filter 1SelectionandClusteringAlgorithmsClock Discipline AlgorithmCombiningAlgorithmPeer 2Filter 2Loop FilterP/F-Lock LoopPeer 3Filter 3TimestampsVFONTP MessagesMultiple servers/peers provide redundancy and diversity.Clock filters select best from a window of eight time offset samples.Intersection and clustering algorithms pick best truechimers and discard falsetickers.Combining algorithm computes weighted average of time offsets.Loop filter and variable frequency oscillator (VFO) implement hybrid phase/frequency-lock (P/F) feedback loop to minimize jitter and wander.
98 NTP subnet configurations
**S4S3S3Workstation(a)Clients(b)S1S1S1S1S1S1***S2S2S2Clients(c)* to buddy (S2)(a) Workstations use multicast mode with multiple department servers.(b) Department servers use client/server modes with multiple campus servers and symmetric modes with each other.(c) Campus servers use client/server modes with up to six different external primary servers and symmetric modes with each other and external secondary (buddy) servers.
99 NTP - Goals and non-goals
Provide the best accuracy under prevailing network and server conditions.Resist many and varied kinds of failures, including two-face, fail-stop, malicious attacks and implementation bugs.Maximize utilization of Internet diversity and redundancy.Automatically organize subnet topology for best accuracy and reliability.Self contained cryptographic authentication based on both symmetric key and public key infrastructures and independent of external services.Non-goalsLocal time – this is provided by the operating system.Access control - this is provided by firewalls and address filtering.Privacy - all protocol values, including time values, are public.Non-repudiation - this can be provided by a layered protocol if necessary.Conversion of NTP timestamps to and from other time representations and formats.
100 NTP Version 4Current NTP v3 has been in use since 1992, with nominal accuracy in the low milliseconds.Modern workstations and networks are much faster today, with attainable accuracy in the low microseconds.NTP v4 architecture, protocol and algorithms have been evolved to achieve this degree of accuracy.Improved clock models which accurately predict the time and frequency adjustment for each synchronization source and network path.Engineered algorithms reduce the impact of network jitter and oscillator wander while speeding up initial convergence.Redesigned clock discipline algorithm operates in frequency-lock, phase-lock and hybrid modes.The improvements, confirmed by simulation, improve accuracy by about a factor of ten, while allowing operation at much longer poll intervals without significant reduction in accuracy.
101 NTP v4 Autonomous System model
Fire-and-forget softwareSingle software distribution can be compiled and installed automatically on most host architectures and operating systems.Run-time configuration can be automatically determined and maintained in response to changing network topology and server availability.Optional autonomous configuration (Autoconfigure)Multicast survey nearby network environment to construct a list of suitable servers.Select best servers from among the list using a defined metric.Reconfigure the subnet for best accuracy with overhead constraints.Periodically refresh the list in order to adapt to changing topology.Optional autonomous authentication (Autokey)For each new server found, fetch and verify its cryptographic credentials.Authenticate each message received using engineered protocol.Regenerate keys in a timely manner to resist compromise.
102 An example of a busy NTP server
NTP primary (stratum 1) server rackety is a server supporting 734 clients all over the world.This machine supports NFS, NTP, RIP, IGMP and a mess of printers, radio clocks and an 8-port serial multiplexorThe mean input packet rate is 6.4 packets/sec, which corresponds to a mean poll interval of 157 seconds for each clientEach input packet generates an average of 0.64 output packets and requires a total of 2.4 ms of CPU time for the input/output transactionIn total, the NTP service requires 1.54% of the available CPU time and generates 10.5, 608-bit packets per second, or 0.41% of a T1 lineThe conclusion is that even a slow machine can support substantial numbers of clients with no significant degradation on other network services.
103 Precision timekeeping equipment (prior to 2000)
Austron 2200A GPS ReceiverAustron 2000 LORAN-C ReceiverSpectracom 8170 WWVB ReciverHewlett Packard 5061A Cesium BeamFrequency StandardNTP primary time server rackety
104 Squeezing the nanoseconds
This shows the residual error measured between the Austron 2201 GPS receiver and the HP 5061A cesium clockThe GPS receiver is stabilized using the LORAN-C receiver, which improves its accuracy to about 50 ns, in spite of the intentional degradation introduced in the GPS signal available to the public
105 NTP resources NTP home: http://www.ntp.org
NTP v3 Specification RFC 1305Simple NTP (SNTP) v4 specification RFC 2030Applicable to IPv4, IPv6 and ISO CNLSList of public NTP time servers (as of Dec 2007)177 active primary (stratum 1) servers278 active stratum 2 serverspool.ntp.org – A a big virtual cluster of timeservers striving to provide reliable easy to use NTP service for millions ofclients without putting a strain on the big popular timeservers.
106 Miscellaneous Application Protocols
Remote Procedure Call (RPC) is a technology that allows a computer program to cause a subroutine or procedure to execute in another address space (commonly on another computer on a shared network) without the programmer explicitly coding the details for this remote interaction. RPC may be referred to as remote invocation or remote method invocation (RMI).Strictly said: RPC is a part of the OSI Session layer.Many different (!) implementations of RPC: ONC RPC, DCE/RPC, MSRPC etc.
107 Miscellaneous Application Protocols (2)
Network File System (NFS) – a protocol for sharing files and directories for access over a network. It is based on RPC. Developed by Sun Microsystems, The current version is 4 (RFC 3530)Common features:Several security mechanismsthe COMPOUND procedureFilesystem Replication and MigrationFile OPEN and CLOSE (net)operationsFile lockingClient Caching and Delegation