This blog is mainly for myself but it may serve anyone who needs a refresher or who is beginning to enter the world of SIP and Skype for Business. Coming from a traditional Microsoft background, when I started with Lync I had no conception of voice, or SIP and spent the best part of my early career actively avoiding anything to do with the subject.
I have never been officially trained on Lync or Voice over IP. All of my knowledge has been a mixture of trial and error, death by fire and reading other people’s blogs. Some might say this is the best way to learn, and while I agree to a point, there are times where I am in a room full of people and feel like the novice still. There are still some areas where I feel I have gaps in my “data bank” as it were. As a result, I often find it difficult to grasp some concepts on how things work outside of Lync / Skype for Business. I have never deployed or administered a Cisco or Avaya PBX system for instance because I just haven’t been in a situation where I have needed to. I am Microsoft, why would I have this experience? But as a Skype consultant you’re expected to have knowledge of these systems to some degree if you are to integrate voice especially.
I know I am never going to deploy or work on these other systems, but learning how the underlying protocols work gives you a firmer footing to deal with questions around it. For instance Cisco and Microsoft are two very different VoIP ecosystems but they can quite easily talk to one another because under the bonnet there is a common set of protocols that define how they fundamentally work. SIP is just one of these protocols. Both Cisco, Microsoft and others conform to at least SIP RFC 3261 an internet standards based protocol. Granted almost every system has their own small modifications to the protocol (usually additional information only relevant to their system within the SIP message) but the core methods are pretty standardised. The result is of course that systems can communicate with each other at various degrees of integration.
Viewing SIP messages in Skype for Business you need the debugging tools installed on your machine. SIP messages are not located in Server Event Logs like most Microsoft applications. They are stored in log files on clients and can also be captured from the Server using centralised logging or using a network packet capture program such as Wireshark.
With Skype for Business Online and Cloud PBX you lose some of this end to end traceability because you don’t have access to the Front End Servers, so all your troubleshooting is done using the client log file called Lync-UCCApi.UCCApiLog located in the local app data folder of the user’s profile. You open this log using Snooper, a tool included in the Skype for Business Debugging Tools. Snooper is a log parse program especially designed to parse Skype for Business SIP log files. It provides deep insight into a SIP conversation and it fundamental to troubleshooting communication problems within the Skype for Business ecosystem.
SIP Messaging 101
When you first open Snooper and parse the log file it can be quite daunting. I still struggle sometimes and get lost. Hence the reason for this blog. The key tab you probably always want to start in is the “Messages” tab. Here, lists all the SIP messages captured by the log. The trace tab gives you verbose information of the entire log file.
The first thing you want to do is filter the log to show only to conversation you want to troubleshoot. You can do this by searching the log for unique values e.g. the called party or find the first invite in the conversation and right click to find related messages to the same conversation. Either way you end up with something like this
This image shows you the complete conversation that happened between two parties. To explain what is going on here, there is a process flow view in Snooper (the purple icon top right with arrows) that provides a graphical view of a conversation
This view lists out the entire message transaction log of the conversation in order. Before we jump into a SIP message and try to make sense of it, lets look at the methods and responses of a normal conversation.
A conversation begins when an endpoint sends an INVITE method to their SIP Proxy (Skype for Business Front End or Edge). The INVITE contains information on who the endpoint is, who they are trying to contact and other information required in order to setup a call such as location information, supported methods and features. The SIP Proxy then forwards the INVITE to the called parties SIP Proxy which then forwards to the called parties endpoint (if available). Once the called parties endpoint accepts the INVITE it responds with a 200 OK message. The calling parties endpoint then sends an ACK message to the called parties client and then the call is setup to allow media to flow.
However, while this is going on, a whole bunch of other stuff is happening to make sure that by the time the calling party sends the ACK, both clients are ready to accept media.
You will notice that between the first INVITE and the ACK, several events happen. Events starting with 1xx are informational messages that provide an update to clients during a conversation. 100 TRYING means that the SIP Proxy is trying to proxy the communication to the called party. 101 PROGRESS REPORT is the result of the TRYING request. 183 SESSION PROGRESS is often referred to as Early Media. Early Media does not mean necessarily that media will be established, but rather the point when media capabilities between endpoints are exchanged. Typically 183 messages contain SDP information which tells each client what codec to use and which media candidate to connect to. Eventually you will see a 180 RINGING message. This is often referred to as Late Media. This is the point where both endpoints have enough information of each other in order to communicate and is often the point where you can guarantee that the called party will hear a ring tone. When the called party picks up the receiver, then the 200 OK message is sent by the called party and the ACK by the calling party. Then bingo!, you have an established call.
When the conversation ends, the endpoint that leaves the conversation first sends a BYE message to the other party. When the other party receives this message it responds with a 200 OK message and the call is then teared down and no longer valid.
Contents of a SIP Message
When you look at the body of a SIP Message it looks similar to e-mail. This is because the SIP protocol is an ASCII based protocol, meaning that it is a human readable format. As a result, people can make sense of what is going on rather than having to convert hex to ASCII to find error codes etc.
This image is an extract from the first INVITE message sent in a conversation. We can see it is an INVITE as the method is clearly displayed in BOLD and contains the SIP address of the called party as well as the protocol being used (SIP version 2.0). The VIA header is the network layer that the client is going to use. In this example we are using SIP version 2.0 encrypted with TLS from 192.168.1.61 which is the calling parties host machine. The port 32528 is a ephemeral outbound port that is used for SIP messages relating to this conversation.
The MAX-FORWARDS header is the amount of times that this message can be forwarded by a SIP Proxy before it is dropped. 70 times is an industry standard maximum. The FROM header contains the identity of the calling party and the TO header contains the identity of the called party exactly like an e-mail.
The CALL-ID header is a globally unique ID (similar to a MAC address) of the conversation. This ID is used to identify all subsequent messages relating to this conversation. The CSEQ header is the Call Sequence identifier. Here 1 is displayed because it is the first message to be sent and the method used is INVITE. This will increment as the conversation continues and is useful to determine where you are in a conversation when troubleshooting.
The CONTACT header contains the identity of the party where responses should be sent to. The USER-AGENT is the physical client the party is using to generate this conversation. In this case we can see that it is Skype for Business version 16.0.7571.2109 which relates to Skype for Business 2016 Click to Run Windows.
The SUPPORTED headers contain information of what other methods and headers the client supports in this conversation. Here we can see that the client supports features like transfer, escalation to conference, media bypass, early media, session timer etc. The MS-Conversation-ID is a Microsoft only header that is used to keep track of the conversation internally. The MS-KEEP-ALIVE header is also a Microsoft header that is used to keep connections alive when SIP is sent over the TCP network transport protocol. Noteworthy mention here is that UAC means User Agent Client e.g. my workstation. If it was UAS this would mean User Agent Server which would be the SIP Proxy server.
The ALLOW header tells the other party what SIP methods the client supports. So we have INVITE, BYE, ACK which we have already talked about, but what about the others?
- CANCEL – this method is used to cancel a pending message / request e.g. you hang up before the called party answers
- INFO – This is used for mid-session signaling information to be passed between clients
- UPDATE – This is used for when something changes in a conversation from the last setup configuration, such as changing codecs due to bandwidth etc.
- REFER – Refer is used to transfer a conversation from one endpoint to another like transferring from your desk phone to your mobile, or to another party entirely, or even voicemail
- NOTIFY – Notify is used to alert an endpoint when the state of another endpoint changes. For instance the presence status of your contacts changes, your client will receive an alert of this in a NOTIFY message via the presence broker (front end server) which handles the SUBSCRIBES (explained later).
- BENOTIFY – This is specific to Skype for Business / Lync. BE NOTIFY or Best Effort Notify is a method that allows notifications to be sent without the receiving endpoint sending an 200 OK and generating an ACK. The result of this is less network traffic. BENOTIFY is used for less important alerts.
- OPTIONS – This allows an endpoint (or user agent) to query the capabilities of another client to discover information about supported methods. This is used in Skype for Business to determine if communication can be established between two endpoints. You will hear the term SIP OPTIONS ping quite regularly as a method SBCs and ISPs use to determine if the gateways are online or not. A reply means up, and timeout means down.
The MS-SUBNET header is unique to Microsoft and this is used for determining the network location of the endpoint. This is different to the SIP Location Server in that this information is used specifically for E-911 within Skype for Business.
The MS-ENDPOINT-LOCATION-DATA header again is Microsoft specific and contains data about where the media will be travelling. In this case over the Internet. The P-Preferred-Identity header is the assumed asserted ID of the caller, until we figure this out. In this case my SIP address. This is used for Caller ID services and if not set correctly can cause problems when dialling over the PSTN.
The PROXY-AUTHORIZATION header is used to authenticate the session through SIP Proxies. The CONTENT-TYPE header tells us that the SIP message also includes an application. This application is SDP or Session Description Protocol which is used for determining Media Codecs and Target Media candidates (available endpoints).
Finally the CONTENT-LENGTH header contains the number of bytes the application consumes.
The session progress report provides the status of the call setup. Again this is referred to as Early Media and to reiterate does not mean that you can speak at this point, but rather a time where both user agents (endpoints) may have enough information about each other to theoretically establish a media stream. 183 can contain SDP information but 180 RINGING does not. For instance, when you call a contact center and it fake rings once or twice, then plays hold music, this is an example of Early Media in effect.
In this progress report we can see that we have a new header called RECORD-ROUTE. These headers contain a list of SIP proxies that the SIP message has traversed in order to reach the called parties user agent. In this conversation both parties are using Skype for Business Online, but the calling party is in the Amsterdam Office 365 data center, while the called party is in the Irish Office 365 data center.
We also begin to find out information about the called user agent. In this case the USER-AGENT header contains information about the called party device, a Yealink T48G Skype for Business Edition phone running firmware version 126.96.36.199. So we now know the endpoint we are going to get through to, but we need to understand it’s capabilities.
The 180 RINGING message is sent when the called party’s user agent has finished exchanging capabilities and understand what audio / video codec to use to communicate with the calling party user agent. Here we can see that the Yealink phone is allowing SIP methods INVITE, INFO, PRACK, ACK, BYE, CANCEL, OPTIONS, NOTIFY, REGISTER, SUBSCRIBE, REFER, PUBLISH, UPDATE and MESSAGE.
Some have been discussed earlier in this post, but the ones we haven’t talked about yet are:
- PRACK – stands for Provisional Response ACKnowledgement. This is like an ACK message but is used to provide a response to the sending user agent that the destination user agent has received a provisional message such as a 183 SESSION PROGRESS message. PRACK stops calls hanging in limbo in the event of a network issue during call setup.
- REGISTER – This message is responsible for retrieving the user agent’s location from the SIP Location Server to find it’s Address of Record (AOR). In our case the Front End server is our registrar and location server and this will respond with the user-agents host IP address e.g. email@example.com. The REGISTER message is important part of multiple points of presence (MPOP) within Skype for Business and allows mobility between subnets.
- SUBSCRIBE – This message is used to susbcribe the user agent to a particular service. This could be a presence broker or perhaps Exchange Unified Messaging (Voicemail) to return a notification of a new message and light up the message waiting indicator (MWI) on a desk phone for instance.
- PUBLISH – Publish works in hand with SUBSCRIBE. When the subscribed to system want to send information to subscribed clients it uses a PUBLISH message. This is commonly used to notify clients of a new voicemail for instance.
- MESSAGE – is the method that allows the transfer of instant messages between user agents. These could by system messages or end user communication.
Back to the 183 message, and we can see that the Yealink phone is sending its allowed features using the ALLOW-EVENTS header. We can see that it is allowing, audio, conference, hold and transfer.
The MS-TELEMETRY-ID header is a Microsoft header. This is used to provide Microsoft with telemetry information about the call and the ID is the telemetry identifier of this call. If you create a user policy that disallows telemetry data to be sent to Microsoft, then this header will not be present. However, for Skype for Business Online, it is on by default.
The MS-EDGE -PROXY-MESSAGE-TRUST header, again is a Microsoft only header. This contains the information about the called party’s Edge server and declares that this conversation can be trusted to traverse the Edge SIP Proxy at the destination without re-authenticating each time.
Now we move on to the SIP 200 OK message. After all the progress messages and call setup information has been discovered and decided, the 200 OK message is sent by the called party’s user agent when the call has been answered.
By now the message should be self explanatory, but we can see that within the 200 OK we now have the P-ASSERTED-IDENTITY of the called party. This will contain authoritative information about the called party’s identity aka Caller ID and is used for things like caller display, call back etc. If you had a SIP trace from the called party side, you would see the P-ASSERTED-IDENTITY of the caller too. If this header is wrong, or in an incorrect format e.g. contains ;ext=4000 at the end of the telephone number, then some PSTN carriers will not allow, or have problems forwarding your call to the intended recipient (e.g. the called party receives a malformed caller display). Therefore, it is good practice to strip any extensions from your identity as the call passes over your outbound SIP trunk from Skype for Business to your SBC.
The calling party’s user agent will send an ACK in response to the received 200 OK message from the called party’s user agent. This is just a simple acknowledgement and is the point where the user agents begin to send media.
When a party hangs up a call, both clients need to know when this happens in order to tear down a call. If this does not happen then the call will stay up until the session timer expires and no progress reports have been received within that time. Session timers can vary depending on the platform, from 30 seconds to 5 minutes I have seen. The impact of not declaring a call as being terminated could mean additional billing from your telephone service provider for consumed minutes.
To combat this the party who hangs up sends a BYE message to the other party. In this example you can see that the Yealink phone was the party who hung up first and therefore sent the BYE message to the Skype for Business client party. The party who receives the BYE message sends an ACK back to confirm it has received and understood the message and the call is then torn down.
Hopefully this post will help you understand how Skype for Business communicates and the fundamentals of how a call is established.