Testing M2M and Consumer RSP beyond the specifications: the Subscription Manager [PART 4]
Written by Iain Maxwell.
Welcome back to the latest instalment of this blog on testing M2M and Consumer RSP entities beyond the specifications. Last week we looked at testing the device for the M2M and Consumer RSP domains. Therefore, this week I will concentrate on testing the Subscription Manager.
Subscription Manager Testing
Let’s now concentrate on the Subscription Manager looking at what is tested and any issues and problems associated with testing the different types of Subscription Managers, to determine if we need to test beyond the specifications, and also how to do this in practice. The following table summarizes the scope of the M2M and Consumer RSP Subscription Manager testing and you can see that both functional and physical operational security testing are required.
Table 1: Subscription Manager Testing Overview
As previously indicated in the first blog in this series both M2M and Consumer RSP have two different types of Subscription Managers:
- M2M has the Subscription Manager for Data Provisioning (SM-DP) and a Subscription Manager for Secure Routing (SM-SR)
- Consumer has the Subscription Manager for Data Provisioning Plus (SM-DP+) and Subscription Manager for Discovery Service (SM-DS)
In M2M for both the SM-DP and SM-SR testing there is currently no certification body responsible for the certification process, but the SGP.11 test specification does provide full functional test coverage for both Subscription Manager types. Then the M2M physical security is delivered by a GSMA SAS audit using 3rd party auditors to deliver SAS-Subscription Manager (SAS-SM) certification against the GSMA’s FS.08, FS.09 & FS.10 security documents.
Again in Consumer RSP for both the SM-DP+ and the SM-DS testing there is currently no certification body responsible for the certification process, but the SGP.23 test specification does provide full functional test coverage for both Subscription Manager types. Then the Consumer RSP physical security is again delivered by the same GSMA SAS audit as performed for M2M.
Subscription Manager Certification
Each of the separate Subscription Manager test areas performed combine together to create the overall Subscription Manager product compliance process required by GSMA for a Subscription Manager to enter the M2M or the Consumer RSP ecosystem.
Figure 1: Subscription Manager Certification Overview
Functional testing is performed first but the reality is it is only ‘self-testing’ as GSMA have not agreed on a Certification body responsible for the functional testing. Next the physical or operational security is performed as a SAS-SM audit at the Subscription Manager vendor’s data centre. The Subscription Manager vendor then submits all the results of the testing and audits that they have performed. The GSMA will then perform an evidence check to analyse the validity of the results. If the evidence check is successful the GSMA CI will then issue and sign the Subscription Manager Certificate’s for authentication and profile encryption to the Subscription Manager vendor. At that point only then can the Subscription Manager vendor declare that their product is fully compliant with the GSMA certification process for M2M or Consumer RSP Subscription Managers.
Subscription Manager Testing: Issues and problems
Let’s first of all consider Subscription Manager testing issues related to the Certification process.
As yet GSMA have not identified a Certification Body capable and agreeable to perform Subscription Manager certification for either M2M or Consumer RSP, although GlobalPlatform and GCF have both been approached by GSMA. GlobalPlatform processes require that the full entity under test is in the test lab during qualification, but Subscription Manager vendors have not been able to comply with this requirement, so unfortunately GlobalPlatform are not looking like a viable option here. GCF have requested more time to consider the scope and implications associated to this task. So this lack of certification may lead to future interoperability issues and debates are still on-going regarding the scope of Subscription Manager certification, for example, does RSP functionality + Subscription Manager OS + hardware constitute Subscription Manager certification? Well although this is still under debate a consensus looks likely within GSMA RSP-CERT where the RSP functionality will be certified independent of the Subscription Manager OS and the hardware. So hopefully that will help the GSMA in their search for a responsible certification body.
Now we move to consider any Subscription Manager testing issues or problems caused by technical details.
In this case the ES2+ interface in Consumer RSP is still out of scope of SGP.23 v1.2, which will continue to cause interoperability concerns between Operators and SM-DP+ vendors. Although ES2+ will be finally agreed in SGP.22 v3.0 due for release later this year, so following that SGP.23 v2.0 will be updated to test this interface between the Operator and the SM-DP+. Other testing issues in SGP.23 caused by inconsistent wording in SGP.22 where the optional use of Icons, Icon types etc. versus mandatory support for notifications, profileOwner, policy rules in profile metadata caused several different Subscription Manager vendor interpretations.
Issues were also discovered in M2M for the SM-SR Change Process which were resolved in the recent update to SGP.11 v3.2 mid 2017.
Subscription Manager Test Tools – solving the problems
To solve these problems, and in general perform the required M2M Subscription Manager testing UL provide 2 test tools. The ‘UL SGP.11 SM-DP Test Suite’ and the ‘UL SGP.11 SM-SR Test Suite’ offer functional testing against SGP.11 for the SM-DP and SM-SR.
Figure 2: UL M2M Subscription Manager Test Suites
These tools can only be used for ‘self-testing’ purposes because GSMA have not managed to find a certification body to take ownership of the Subscription Manager certification process.
The pink area in each of the tools is the part of the architecture that the tool is simulating in order to perform the testing. So for the SM-DP the ES2, ES3 and ES8 interfaces are the active Subscription Manager interfaces that are tested. Then for the SM-SR the ES1, ES3, ES4, ES5, ES7 and ES8 interfaces are the active Subscription Manager interfaces that are tested.
Then, again to solve these problems and in general perform the required Consumer RSP Subscription Manager testing, UL provide 2 test tools. The ‘UL SGP.23 SM-DP+ Test Suite’ and the ‘UL SGP.23 SM-DS Test Suite’ offer functional testing against SGP.23 for the SM-DP+ and SM-DS.
Figure 3: UL Consumer Subscription Manager Test Suites
As mentioned before these tools can only be used for ‘self-testing’ purposes because GSMA have not yet managed to find a certification body to take ownership of the Subscription Manager certification process.
The pink area in each of the tools is the part of the architecture that the tool is simulating in order to perform the testing. So for the SM-DP+ the ES8+, ES9+, ES11 and ES12 interfaces are the active Subscription Manager interfaces that are tested. Please note that ES2+ is still out of scope. Then for the SM-DS the ES11, ES12 and ES15 interfaces are the active Subscription Manager interfaces that are tested.
Subscription Manager Audit Services – solving the problems
Physical security offered by GSMA SAS-SM has been adopted by the Subscription Manager worldwide and offers more than adequate safeguards.
Currently UL do not offer SAS-SM as GSMA have selected a preferred set of auditors and will not open up the tender process again until 2020.
Subscription Managers should be fully testing their products against the functional testing defined in SGP.11 for M2M and SGP.23 for Consumer and be able to demonstrate that they have performed this functional testing. But the Subscription Manager certification process for M2M and Consumer RSP relies on self-certification by the Subscription Managers themselves, which inherently is not a reliable approach.
There is no chosen certification body to govern the process and there does not seem to be a viable alternative at least until the long-term, but UL believes that GSMA could mandate in SGP.24 for Consumer RSP or the equivalent SGP.24 process document from M2M, the use of a third party test tool which would surely help to improve the process.
Subscription Managers have many optional features that they may or may not support and it is expected that many Subscription Manager vendors will choose to test only the features that they need for their given customers. So it is advised that any customers of the Subscription Manager vendors pay special attention to which optional features have been tested by the Subscription Manager to ensure that they get the feature set that they as a customer are expecting.
Therefore it is recommended to test the Subscription Manager beyond the SGP.11 and SGP.23 test specifications to identify and resolve any possible interoperability issues such as SM-SR Change Process for M2M or potential ES2+ issues for Consumer RSP respectively.
Next week we will focus on debugging issues related to the eUICC, device and Subscription Manager before we finally reflect and analyse the findings. So please join me then for the final conclusion to this series of blogs.