Testing M2M and Consumer RSP beyond the specifications: The eUICC [PART 2]
Welcome back to this blog on testing M2M and Consumer RSP entities beyond the specifications. Last week we looked to the two different architectures for M2M and Consumer RSP to identify the different entities ‘in-scope’ for testing in each architecture. Therefore, this week I will concentrate on testing the eUICC.
Let’s now concentrate on the eUICC looking at what is tested and any issues and problems associated with testing it, to determine if we need to test beyond the specifications, and also how to do this in practice.
Table 1: M2M and Consumer eUICC Testing Overview
This table summarizes the scope of the M2M eUICC testing on the first 4 rows. You can see that the testing is split into functional, eUICC profile, logical security and physical or manufacturing related security testing.
- Functional M2M eUICC testing is delivered by GlobalPlatform following the GSMA’s SGP.11 test specification.
- M2M eUICC Profile testing was planned to be delivered by GlobalPlatform, but GlobalPlatform didn’t manage to qualify any test tools or eUICCs against the SIMalliance eUICC Profile Test Specification v2.0 for M2M at the GlobalPlatform test fest in December 2017. So that will hopefully be delivered in early Q3 2018 as an add-on during the M2M v3.2 GlobalPlatform test fest.
- Logical security is delivered as a SOG-IS SGP.05 audit for ELA4+ Common Criteria against the M2M eUICC Protection Profile.
- M2M physical security is delivered by a GSMA Security Accreditation Scheme (SAS) audit using 3rd party auditors to deliver SAS-UICC Production (SAS-UP) certification against the GSMA’s FS.04, FS.05 & FS.06 security documents.
Then for Consumer RSP eUICC testing it is again split into the same areas as M2M:
- Consumer RSP functional eUICC testing is delivered by GlobalPlatform following the SGP.23 test specification.
- Consumer eUICC Profile testing will be delivered by GlobalPlatform following the SIMalliance eUICC Profile Test Specification v2.1.
- Logical security is temporarily delivered as a SOG-IS Security IC Platform Protection Profile audit PP084, until the SGP.25 audit for ELA4+ Common Criteria against the Consumer RSP eUICC Protection Profile is ready.
- Consumer RSP physical security is delivered by same GSMA SAS-UP audit as M2M.
You should also note that both M2M and Consumer eUICCs are also tested in accordance to 3GPP and ETSI test specifications.
Each of the separate eUICC test areas performed combine together to create the overall eUICC product compliance process required by GSMA for an eUICC to enter the M2M or the Consumer RSP ecosystem.
Figure 1: M2M and Consumer RSP eUICC Certification Overview
Functional testing is performed first but the reality is that it goes hand in hand with the eUICC Profile testing. Next the logical security audit is performed and then the physical security SAS-UP audit is performed at the EUMs manufacturing site. The EUM then submits all the results of the testing and audits that they have performed, including the GlobalPlatform product qualifications and the result of the Common Criteria audit against the relevant Protection Profile. The GSMA will then perform an evidence check to analyse the validity of the results. If the evidence check is successful the GSMA CI will then issue and sign the EUM Certificate to the EUM so that they can generate and sign the necessary eUICC Certificates required to be loaded onto each individual eUICC. At that point, only then can the EUM declare that their product is fully compliant with the GSMA certification process for M2M or Consumer RSP eUICCs.
Please note that as mentioned previously alongside all of these RSP type certification activities, both M2M and Consumer RSP eUICCs are also tested in accordance to 3GPP and ETSI test specifications including any associated qualification or validation that is mandated by the applicable certification bodies.
eUICC Testing: Issues and problems
Generally speaking eUICC testing is in a good state for M2M and for Consumer RSP, but there are a few functional issues and also issues related to the Certification process.
In practice functional M2M eUICC qualification testing by GlobalPlatform is not being requested or mandated by the MNOs, so the EUMs are in general not functionally qualifying their products. Therefore to combat this for Consumer RSP eUICCs GSMA created SGP.24, which defines the complete certification process for all entities under test including the eUICC. Also testing to SIMalliance Interoperable Profile Test Specification v2.0 for M2M is not mandated by the MNOs either, so this could also lead to future interoperability problems. Currently it is clear that M2M needs an SGP.24 equivalent type specification to define the scope and processes required for M2M eUICC certification.
GSMA are in the process of creating the SGP.24 equivalent for M2M eUICC but until that is completed and there is buy in from the MNOs to mandate such a document when it is available there is no real guarantee that full interoperability is actually being achieved by all eUICCs entering the M2M or Consumer eUICC ecosystems.
Although the certification path for Consumer eUICCs is quite clear, there is currently only one EUM with a fully GlobalPlatform qualified product, and even with GSMA’s SGP.24 certification process specification in place, if the MNOs and MVNOs do not unite to mandate this process the same issues that befell the certification of M2M eUICCs might come back to haunt the Consumer world.
Now we move to consider any eUICC testing issues or problems caused by technical details.
In this case, previously unclear or frequent versions of SGP.22 has already introduced interoperability issues that needed fixed in SGP.22 and SGP.23, e.g. the fact that eUICC profileClass may or may not be sent depending on the SGP.22 version that the eUICC supports, or the need for the use of individual Store Data chained sequences for each of the individual parts of the bound profile package that is downloaded, i.e. InitialiseSecureChannel, ConfigureISDP, StoreMetadata, ReplaceSessionKeys and LoadProfileElements, instead of one Store Data chained sequence for all of the individual parts of the bound profile package combined.
Also, UL have observed that there is less comprehensive negative test coverage in SGP.11 as compared to SGP.23, therefore in the future SGP.11s coverage could be improved. Plus there are still many excluded CAT_TP test cases in M2M Functional test tools – but this lack of full test coverage might impact those MNOs who chose to support CAT_TP.
Then you should note that M2M relies on GlobalPlatform & ETSI/3GPP test specs and any lack of testing or qualification against these specs could also lead to future OTA interoperability issues.
Finally the SOG-IS SGP.25 audit for ELA4+ Consumer eUICC Protection Profile is not yet available, so the Security IC Platform Protection Profile audit PP084 is in place until GSMA’s SGP.05 equivalent is available. Which hopefully will be adequate until the Consumer RSP protection Profile is available, but some of the logical security checks might be missed or might not be fully in place until this happens, and even then many of the EUMs feel that the Common Criteria approach to certification is too ‘heavy-handed’ for the task in hand. As such these same EUMs have avoided performing these time consuming and expensive audits because they have been able to convince their MNO or even OEM customers that their product security is sufficient. This has in turn caused the GSMA to create a sub working group to investigate how to make the security audit more versatile and quicker whilst delivering almost equivalent security guarantees. The ETA for this is pretty tight but it isn’t expected to be fully up and running until at best late Q4 2018.
eUICC Test Tools – solving the problems
To solve these types of problems and in general perform the required testing, UL provides a range of eUICC test tools.
The ‘UL GlobalPlatform eUICC M2M Compliance Test Suite’ and the ‘UL GlobalPlatform eUICC Consumer Compliance Test Suites’ are two separate test suites that offer GlobalPlatform functional compliance testing against SGP.11 and SGP.23 respectively.
Figure 2: UL GlobalPlatform M2M and Consumer eUICC functional test suites
The pink area in each of the tools is the part of the architecture that the tool is simulating in order to perform the testing. For M2M, the ES5, ES6 and ES8 interfaces are the active eUICC interfaces that are tested. Then for Consumer RSP the ES6, ES8+, ES10a, 10b and 10c interfaces are the active eUICC interfaces that are tested.
Using these official GlobalPlatform compliance test suites UL also offers official eUICC qualification services in our GlobalPlatform qualified lab.
Then the ‘UL SIMalliance eUICC Profile Test Suite’ offers SIMalliance eUICC Profile GlobalPlatform compliance testing against the SIMalliance eUICC Profile Test Specification v2.0 for M2M eUICCs and also v2.0 and v2.1 for Consumer RSP eUICCs. Currently GlobalPlatform has only qualified Consumer eUICCs against the SIMalliance eUICC Profile Test Specification v2.1, but M2M eUICC will be fully qualified against the SIMalliance eUICC Profile Test Specification v2.0 by end of Q2 2018. In the meantime M2M eUICCs are self-tested by the EUM.
Figure 3: UL GlobalPlatform M2M and Consumer eUICC SIMalliance profile test suite
The pink area again in each of the tools is the part of the architecture that the tool is simulating in order to perform the testing. So for M2M the ES5 and ES8 interfaces are the active eUICC interfaces that are tested when loading and managing profiles. Then for Consumer RSP the ES10a, 10b, 10c and ES8+ interfaces are the active eUICC interfaces that are tested when loading and managing profiles.
Again using this official GlobalPlatform compliance test suites UL also offers official eUICC qualification services in our GlobalPlatform qualified lab.
Finally, UL also offer the ‘UL Mobile eUICC Profile Tester’ which is an ‘ad-hoc’ test tool that allows full eUICC profile management for M2M and Consumer RSP eUICCs to be tested. As a bonus it also verifies and validates that after a loaded SIM profiles is enabled, that the full SIM profile is installed and available as required; for example, the SIM profile structure, file contents and keys are all correct and as expected. All of this can be tested without the need for a real Subscription Manager, Mobile Network or even a device as they are all simulated by the tool.
Figure 4: UL eUICC Profile Management and Profile Validation test Tool
For M2M the ES5 and ES8 interfaces are the active eUICC interfaces and for Consumer RSP the ES10a, 10b, 10c and ES8+ interfaces are the active eUICC interfaces that are used.
eUICC Audit Services – solving the problems
UL offers the SOG-IS SGP.05 audit for Common Criteria ELA4+ against the M2M eUICC Protection Profile. Ul will also offer SOG-IS SGP.25 audit for ELA4+ Common Criteria Consumer eUICC Protection Profile - if GSMA make it available.
But UL are also involved in the GSMA micro group involved in defining the new lightweight logical security audit aimed at replacing the SOG-IS SGP.05 and SGP.25 Common Criteria audits.
Currently UL do not offer SAS-UP as GSMA have selected a preferred set of auditors and will not open up the tender process again until 2020.
The certification process for M2M eUICCs is still not fully mandated and until the equivalent SGP.24 process document from M2M is available EUMs should be fully testing their products against the functional testing defined in SGP.11 and also the SIMalliance Interoperable Profile Test Specification v2.0. UL feels that there isn’t any pressing need to test beyond the specifications from an M2M perspective if the EUM can at least demonstrate that they have performed the functional testing. The only optional addition might be to test the BIP performance of the eUICC during the SGP.11 functional testing to determine if there are any scenarios that might leave the Service Provider using the eUICC in their device without connectivity for any unacceptable periods of time. But if an EUM cannot at least provide evidence of having performed the functional testing then UL advises to stay clear.
When it comes to Logical Security the GSMA SGP.05 SOG-IS CC audit is more than enough to deliver the confidence that the inherent security of the eUICC is robust and in place, but if the EUMs are rejecting it, then it’s up to GSMA to resolve this as soon as possible. Even for M2M, the eUICC is still fairly early in its product lifecycle, so any security breaches will be quite harmful. Therefore UL recommends some additional care be taken during the logical security testing that many of the EUMs are delivering as a self-assessment, especially since they may try to leverage and demonstrate equivalent security via other industry audits that are not explicitly aimed at the M2M or Consumer RSP architecture and ecosystem. But at least the physical security offered by GSMA SAS-UP has been adopted by the EUMs worldwide and offers more than adequate safeguards.
Again it’s a similar story for Consumer eUICCs, so if EUMs can demonstrate that they have performed the equivalent functional testing defined in SGP.23 and also the SIMalliance Interoperable Profile Test Specification v2.1 has been completed by following GSMA’s SGP.24, UL feels that there isn’t any pressing need to functionally test beyond the specifications from a Consumer perspective.
When it comes to Logical Security though, the GSMA SGP.25 SOG-IS CC audit isn’t ready, so again GSMA need to resolve this as soon as possible as Consumer eUICC is at the very early stages in its product lifecycle so any security breaches will be extremely harmful and the same recommendations for M2M apply. But at least again the physical security offered by GSMA SAS-UP has been adopted by the EUMs worldwide and offers more than adequate safeguards.
Next week we will focus on all the testing issues related to the device. So please join me then for some more deep dive analysis.