Version 54 (modified by dburgess, 2 years ago)

--

RELIEF 12-2

This is the coordinating page for OpenBTS operators at  RELIEF 12-2, Camp Roberts, Paso Robles, CA, 29 Feb - 2 Mar 2012.

Quick background: RELIEF is an ongoing program in the US Dept. of Defense (DoD) to improve interaction and cooperation between DoD and non-DoD disaster response teams.

To edit this wiki, you can login as user "guest" with the password "guestpass". If you are a regular editor of this wiki, please request a personal account from Kurtis (log in to see info) so that your changes can be tracked.

OpenBTS operators:

With assistance from:

Favorite toys:

Other participants:

Experiment Plans

Range Networks

Official experiment plan: "Lightweight and Resilient Cellular Networks"

We will operate a light-weight, portable 2.5G cellular network based on OpenBTS software. Our primary interest in this deployment is to gain experience in connections to Ushahidi servers and other open source packages used by the response community. We will also run experiments with RRLP (the query protocol for embedded E911 GPS receivers), handover and GPRS medium-rate data support if conditions allow.

Nuts and Bolts of Range's Stuff

Range OpenBTS cell sites present 2G GSM handsets as SIP clients in an IP network, supporting SMS and telephony. The available interfaces are:

  • Telephone Calls
    • SIP/RTP VoIP, including use of VoIP providers for PSTN gateway service (ie, calling to and from the "real world")
  • SMS
    • SIP-SIMPLE SMS to other GSM/SIP clients and local applications (like sending SMS from one local phone to another)
    • mapping of MO-SMS to HTTP POST transactions
      • for outbound SMS through bulk gateways (like Clickatel)
      • for direct connection of SMS to local applications (like Ushahidi)
    • SIP-SIMPLE SMS to PLMN messaging gateways (ie, texting to and from the real world) like Tropo
  • RRLP, the protocol used for talking to handset-embedded GPS receivers
    • current interface is just an sqlite3 database. If someone want to map that, that would be spiffy.
  • short message cell broadcasting (SMSCB)
    • current interface is just an sqlite3 database
  • GPRS medium-rate data (2.5G)
    • handset appears as a NAT client behind the BTS IP address

These units were operated in the DCS1800 band under experimental license WF9XKG.

Tethr

Official experiment plan: "Crisis Mapping & Communications"

We will operate a crisis mapping and communications solution that provides Wi-Fi access to client computers, tablets and smart phones. This unit can also provide internet backhaul to these clients either via 3G/4G radio or satellite. We will provide a local crisis mapping solution (Ushahidi) accessible to all client devices.

Lastly, we will interface with the Range Networks "Lightweight and Resilieng Cellular Networks" experiment. We will receive SMS messages from cell phones on the cellular network into the Ushahidi crisis map, allowing any cell phone user to create a crisis report via SMS.

Tethr Details

Wi-Fi

Tethr will be providing 802.11n Wi-Fi in the 2.4GHz range. We plan to have at least one SSID available and possibly two. If we end up with a single SSID it will be "tethr". Current plan is to provide this to a large area either with an omni or sector antenna. Backhaul for this Wi-Fi will be either via 3G HSPA/HDSPA networks (AT&T or T-Mobile), or through BGAN IP satellite network. DHCP addressing for Wi-Fi clients will be a private network on the 172.20.12.0/255.255.255.0 network.

Ushahidi Crisis Mapping

Once connected to the Wi-Fi, devices can then access the Ushahidi crisis map running locally on the Tethr box. Events can be added to the crisis map from a web browser. iOS and Android users can access this map via apps on their devices as well as web browsers. SMS can be used (see below) to send a message directly to Ushahidi which will result in a new event being added to the crisis map.

SMS

The Tethr box will utilize a direct SMSC connection using Tropo to send and receive SMS messages from the Range Networks GSM network.

IVR

Using Tropo, the Tethr box can also receive incoming calls from the Range Networks GSM network and provide an IVR menu through which new events can be added to the Ushahidi crisis map.

Other Services

Range Networks Central Server

Range Networks will be running a central server on a public IP address with the following services:

  • Asterisk - SIP PBX.
  • smqueue - SMS store-and-forward server
  • subscriber registry - provisioning and authentication database
  • Tropo/Prism - speech and messaging application server

This central service will provide PSTN/PLMN interconnection through:

By running this server at a known, stable IP address we greatly simplify the problem of connecting calls and routing message between OpenBTS units in different IP subnets and connecting to the public internet through different backhaul paths.

Tropo Call and SMS Routing

Tropo will be providing routing services for calls and SMS to connect OpenBTS systems to the PSTN and PLMN (ie, the "real world"). A key feature of their service is "NAT-for-phone-numbers". We used this routing system for Burning Man 2011 and RELIEF 12-1 and it has great utility for temporary networks operated without incumbent cooperation. It works like this for outbound calls:

  1. OpenBTS customer calls a phone number in the real world.
  2. Tropo pulls a DID from a pool to be used as the caller ID for this call and records the {caller, called, DID} triplet. As long as the triplet is unique, the same DID can be used for many calls by many people.
  3. Tropo routes the call to the real world using the selected DID as the caller ID.

The triplet recorded in the database allows the remote party to return a call to the caller ID DID. Tropo can then route the call back to the original outbound caller using the recorded triplet.

The same procedure can be applied to SMS.

Connecting Stuff

Connecting OpenBTS SMS

Via Public SMSCs

Public SMSC are a convenient interface for applications that normally process SMS inputs, however the use of outside SMSCs is a security risk in situations where the network cannot be trusted.

Tropo

Tropo servers in the RELIEF network can provide bi-direcitonal SMS to and from public SMSCs using the techniques described in "Tropo Call and SMS Routing". These text messages will appear to the receive like text messages from any other cellular network and replies will be delivered to the sending handset.

Clickatel

OpenBTS units in the RELIEF network can send text messages via the  Clickatel service. These messages appear like normal text messages at the receiving end, but all of them will show the same originating number and there is no reply path.

Direct (non-SMSC) Connections

Avoiding public SMSCs can improve both the reliability and security of SMS applications.

HTTP/S

With a little hacking in smqueue, text messages sent to specific numbers can be delivered to applications via HTTP GET methods.

Connection OpenBTS Speech Calls

PSTN Connections

Tropo

Tropo servers in the RELIEF network can provide PSTN origination and termination using the techniques described in "Tropo Call and SMS Routing".

Link2VoIP & Gafachi

These are commercial VoIP/PSTN gateway providers. These providers can be used for outbound routing of any call but inbound routing will only work for a small set of pre-provisioned handsets.

Local SIP Connections

Normal Speech Calls

Any local SIP endpoint can be assigned an extension for local call routing. If the handset is provisioned with a telephone number, calls can be bi-directional.

Emergency Calls

Emergency calls get special-case routing. These calls can be routed to any SIP endpoint, including an on-site simulated PSAP or TOC.

RRLP

OpenBTS nodes support RRLP, the protocol used to communicate with GPS receivers in E911-compliant handsets. The information is current just dumped into an sqlite3 database on each BTS unit. Remote access to this information would be simple though a CGI program running in the web server on each unit, but someone would need actually write that in time for the exercise.

Topologies

Sites

Sites of primary interest (stuff we need to connect):

  • MedWeb truck: MedWeb server(s), VSAT terminal, one 10W rack-mount OpenBTS unit on push-up mast. This site has public IP addresses.
  • Tethr ops center: Tethr server, BGAN terminal, one 1W pole-mount OpenBTS unit on push-up mast.
  • Range ops center: BGAN terminal, one 1W pole-mount OpenBTS unit on push-up mast, one 100 mW OpenBTS desktop unit.
  • Range central: server at public IP address running Asterisk, sipauthserve, smqueue, Prism and Tethr. This site has public IP addresses.

A BGAN site gets a single public IP address via DHCP and the local subnet is NAT'd behind that address.

Please do not publish the public IP addresses associated with any of these sites.

Sites of lessor interest (stuff we might connect if there's time and interest):

  • Lockheed-Martin's COW: various 2G/3G/4G systems plus a SIP switch of some kind. IP addressing unknown.

Traffic Types

  • Ushahidi SMS. These are SMS messages to and from Ushahidi servers, entering smqueue as SIP SIMPLE and relayed to Ushahidi via an HTTP short code.
  • Medical SMS. These are application-specific SMS messages for MedWeb, entering smqueue as SIP SIMPLE and relayed to MedWeb servers via an HTTP short code.
  • Normal SMS. These are handled locally by smqueue and gatewayed to/from the PLMN !by Prism/Tropo/Voxeo?.
  • Normal Speech. Speech calls are switched by local Asterisk servers, relayed as needed to Range central and gatewayed to/from the PSTN by Prism/Tropo/Voxeo? or Link2Voip via Range central.
  • SOS Speech. Emergency calls are routed to local PAP2 adapters or designated cellphones via a local Asterisk server.

Topology #1: All Together Now

At McMillan Field:

  • Tethr ops center
  • Range ops center

At God-knows-where:

  • MedWeb truck

In San Francisco:

  • Range central

In this topology, all of the Tethr and Range equipment is tied into a common subnet (we'll call "McMillan") with AirMax bridges. McMillan, MedWeb and Range central connect through the pubic internet via satellite.

Ushahidi SMS Routing

Ushahidi SMS at McMillan goes to the local Tethr server via HTTP. MedWeb Ushahidi SMS goes to the Tethr server at Range central via HTTP. These servers sync up whenever they are in contact.

Medical SMS Routing

All medical SMS goes to MedWeb via HTTP to their public IP address.

Normal SMS Routing

All non-short-code SMS gets relayed by local smqueue instances to the smqueue server at Range central.

Normal Speech Routing

All normal speech calls are routed in the local subnet when possible, otherwise relayed to Range central.

SOS Speech Routing

Emergency speech calls are routed to SIP endpoints within the local subnet.

Topology #2: Range Takes a Field Trip to the FOB

At McMillan Field:

  • Tethr ops center

At the FOB:

  • Range ops center

At God-knows-where:

  • MedWeb truck

In San Francisco:

  • Range central

In this topology, each site is a different NAT subnet and they all connect through the pubic internet via satellite.

Ushahidi SMS Routing

Ushahidi SMS at Tether ops center goes to the local Tethr server via HTTP. All other Ushahidi SMS goes to the Tethr server at Range central via HTTP. These servers sync up whenever they are in contact.

Medical SMS Routing

All medical SMS goes to MedWeb via HTTP to their public IP address.

Normal SMS Routing

All non-short-code SMS gets relayed by local smqueue instances to the smqueue server at Range central.

Normal Speech Routing

All normal speech calls are routed in the local subnet when possible, otherwise relayed to Range central.

SOS Speech Routing

Emergency speech calls are routed to SIP endpoints within the local subnet.

Actual Event

"The best-laid plans of mice and men..." Actually, things went pretty well over all.

OpenBTS operators:

  •  Range Networks supplied the OpenBTS-based GSM basestation equipment.
  •  Tethr brought their integrated Ushahidi-WiFi? node and a BGAN terminal.
  •  MedWeb brought an integrated system with embedded GSM basestation functionality, implemented with a Range OpenBTS "OEM kit". They also brought a VSAT van and more BGAN terminals.

With technical assistance from:

  •  Tropo, who provided routing services for speech calls and text messages to and from the outside world.

The weather reports are important because we spent the entire time outside.

Every day began with a briefing at 0900 where we all reviewed safety notices and each team announced their intended experiments. Every day ended with a "hot wash" where each team summarized the day's activities and had the option to present a brief slide set. On the first two days the hot wash was at 1700. The third day was a half-day with the hot wash at 1200 and most teams leaving Paso Robles immediately afterward. Every day, we needed to clear the area no later than 1800 since that's when the most of the access gates to the base closed. Most of the teams meet for dinner and socializing on the first two nights. (Since Paso Robles is in a very good wine region, and since RELIEF always draws an interesting cross-section of participants, these dinners are not to be missed.)

Day 1

Day 1 was cool and windy with a few periods of heavy rain. The vehicles were filthy by the end of the day from being driven on dusty roads while they were still wet from the rain.

The original plan was to run over as many as three separate satellite backhauls, but our three groups were placed together on the airfield, so we decided to use Medweb's VSAT van for everything. Then we discovered that a nearby (~20 meters) radar was overwhelming and de-sensing the VSAT receiver. We attempted to set up Tethr's BGAN terminal but were unsuccessful in pointing it, so we set up with Medweb's BGAN terminal instead.

On McMillan Airfield

Our main area of activity was at McMillan airfield: the McMillanBTS installation (on the mast), Medweb's satellite truck, and the Range/Tethr solar-powered "operations center", which is a couple of tables a pair of tents.

Rain clouds cleared by sundown on Day 1.

On the Hill Above McMillan

Range deployed another BTS on a nearby hilltop ("HilltopBTS", 1 km away). This was a self-supporting site powered by a 180 W PV panel. HilltopBTS was connected back to the main site at McMillan via a pair of Ubiquity NS5 IP radios. The entire kit for this site, including a solar panel and tools, fit in the back of a station wagon (what actual English speakers call an "estate car").

IP Routing Configuration

Range and Tethr set up a self-contained NAT subnet and backhauled that through Medweb's BGAN terminal (US$5/MB!).

VoIP Routing Configuration

The orange "(*)" logos are  Asterisk servers.

The Asterisk inside McMillanBTS was designated as the central switch for the OpenBTS network. We ran a GSM-over-IAX trunk from the McMillanBTS Asterisk to another Range Asterisk server in San Francisco. This saved a lot of bandwidth relative to simple G.711-over-RTP. The key characteristic of this network design is that all of the elements on the local side of the BGAN terminal form a fully functional GSM/WiFi/VoIP/SMS network whether the BGAN link is available or not. Local calls routed locally and SMS was either delivered locally or stored until the backhaul was available. All PSTN calls were run through Tropo via the Range SF Asterisk server. Tropo-SMS integration had a few lingering bugs but was fixed by the end of the day.

Medweb also brought a an integrated package incorporating

  • a GSM network-in-a-box based on OpenBTS and the Range RAD1 radio
  • a battery and solar charge controller
  • an integrated IVR system based on Asterisk and
  • other Medweb medical informatics applications.

A key feature of this device is that it translates IVR inputs into formatted text messages that can be stored for later delivery if the satellite backhaul fails. This store-and-forward feature was possible because of the self-contained design of OpenBTS.

(We need a photo of the Medweb box.)

We connected the Medweb box into the Range-Tethr subnet and attempted to forward calls from the McMillanBTS Asterisk to the Medweb IVR Asterisk, but could never get the two Asterisk servers to correctly exchange DTMF/keypress information. (It's hard to do IVR with no keypress events.) We tried a second configuration where the Medweb Asterisk was used as the central switch, forwarding most calls to the McMillanBTS Asterisk, but there was a signaling failure there as well.

By the end of the day, Medweb had also made some changes to the VSAT configuration to allow it to operate in the face of the radar interference.

Lessons Learned

  • Always have a backup satellite link, even if it's BGAN.
  • BGAN terminals seem to have a significant failure rate. It's good to bring two.
  • We need to understand more about how to place and manage VSAT dishes.
  • DTMF is not reliable between Asterisk servers, or at least darned tricky to configure. We'll try FreeSwitch next time.
  • Shade structures are not "rain structures". They leak a lot.
  • Juan, from  Synergy Strike Force is an excellent cook.

Day 2

Day 2 was colder, drier and very windy. This slowed us down a little when our hands got too cold to type.

On McMillian Airfield

Less rain, but colder.

Aaron hard at work on receiving SMS messages from OpenBTS using Tropo. We were able to get SMS messages received on the Tethr box into Tropo but in the end were still unable to get those SMS messages posted as crisis reports in Ushahidi.

IP Routing Configuration

Since the Medweb VSAT was now available (and much less expensive than BGAN), we rearranged our IP network to use it for backhaul. To make McMillanBTS visible to the Medweb box, we DMZ'd it through the NAT, exposing it on an IP address in the Medweb subnet.

VoIP Routing Configuration

To support the Medweb experiment, we configured:

  • all calls on the Medweb cell to initially go to the Medweb Asterisk,
  • the Medweb Asterisk to forward non-IVR calls to the McMillanBTS Asterisk and
  • calls on all other cells to go to the McMillanBTS Asterisk.

This was a compromise configuration that allowed Medweb to run its IVR application in their own cell while still routing other outbound calls through the McMillanBTS Asterisk. Inbound calls still went directly from the McMillanBTS Asterisk to the Medweb instance of OpenBTS. To prevent phones on the Medweb cell from constantly re-camping to the other cells, we put a different MNC and MCC on the Medweb cell, used a different IMSI-matching expression for open registration and burned Medweb their own set of SIMs.

Lessons Learned

  • Money spent on a heavy-duty shelter is not wasted.
  • A NAT firewall router with DMZ support is a really handy thing in a field test kit. You never know when you'll have to hang off someone else's subnet.
  • John from  STAR-TIDES shared his private stash of  Chronic, a local Paso Robles wine. We owe him an equally good bottle of wine to help replenish it.

Day 3

The weather was perfect, but this was just a half-day with the event ending after the final hot wash at noon.

On McMillian Airfield

The Lockheed Martin Stalker UAV. We were supposed to integrate with their GSM payload but they could not fly it on Day 2. They did fly the payload on day 3 (pictured here) but we were unable to get connected to their network to exchange SIP traffic. (We tried bridging via Wifi, but it never really worked.) We could have easily connected these two systems at hermes, on a public IP address, but the whole point of the exercise was to minimize dependence on the satellite backhaul.

On the Hill Above McMillan

We lost IP contact with HilltopBTS for a little while and had to replace its NS5 to restore it. That was the first time we have had an outright failure of an NS5 and we need to investigate. That said, these particular units have been to Burning Man three times so far. Once the IP link was restored, we measured the RSSI from HilltopBTS at -77 dBm at the McMillan site: very strong coverage.

Lessons Learned

  • We need more practice at running wifi bridges and should probably just pack an appliance to do it. If Lockheed's COW were closer, we would have just run a cable.
  • Spares, spares, spares. Space spent packing spares is not wasted.
  • McMillan airfield closes at 1700 on Fridays.
  • Always double-check cargo tie-down straps yourself. We lost $1500 worth of PV panels and caused a minor traffic incident on the way home, probably because of an improperly set strap.
  • In future deployments at RELIEF, we need to move the two Range OpenBTS sites further apart, or repoint the hilltop sector antenna to cover a more remote part of the camp. One kilometer is not enough distance to make for a meaningful 2-cell test because there is so much coverage overlap, even with short masts in the 1800 band.

Attachments