Application Programming Interfaces or APIs have existed for decades. However, they have recently seen a significant expansion in usage from their traditional use within areas such as Business to Business (B2B) or partner integrations into several new areas. These include the growing use of APIs that are exposed publicly for developer integrations with cloud platforms in particular, as well as supporting new web application design paradigms such as microservice architectures and Single Page Applications (SPAs). API calls now represent as much as 83 percent of web traffic, according to traffic review detailed in the Akamai State of The Internet Security Report 2018.
This growing ubiquity means that API security is more important than ever. We will look in this article in more detail at the specific issues surrounding API secrets management: the generation, distribution, storage and revocation of credentials and other sensitive configuration variables relating to APIs.
Secrets is a general term used in cybersecurity to describe a broad spectrum of information and data that is intended for a restricted audience only. In general principle, secrets applies to the content rather than type of data. in can refer in a general sense to subsets of any of the three main components of a web application or API: its application code (implemented functional code and business logic); its configuration (variables specifying how the code is implemented on a given environment); and its hosted data (the information processed by the system on behalf of the system’s users or customers).
In specific relation to API secrets management, however, the term refers most commonly to the API’s configuration variables. The most well-known example of a secret configuration variable is a password, as used to authenticate claims to user identity. However, modern APIs may incorporate various different authentication mechanisms relying on different types of shared secrets – including API keys (used to identify the source of an API request), authorization tokens (used to make API requests in support of a user) and private certificates – as well as variables used to secure data transport (such as encryption keys) and other sensitive environmental configuration data such as database connection strings.
The majority of risks to API secrets involve the breach of confidentiality guarantees. Since API secrets are used largely to either authenticate a claimed identity, or else to encrypt data during storage or transmission, an attacker who is able to gain access to API secrets can either directly or indirectly (via assuming another user’s identity) access data that was not intended for them.
The risk of sensitive data becoming compromised and unauthorized agents gaining access to critical resources can be strongly mitigated against by ensuring that best practice principles are followed in order to assure protection at every phase of a secret’s lifecycle, from initial secret creation or generation to their ultimate revocation or deletion.
By and large, best practice approaches to API secrets management involve ensuring that secrets are managed under a well-documented scheme that incorporates both administrative and technical controls, as we will see below. In line with broader best practice trends within information technology – and driven often by the challenges of rapidly scaling platforms and cloud computing – API secrets management overlaps substantially with practices commonly adopted within infrastructure automation management (IAM) and configuration management (CM). These approaches generally involve moving from a series of ad hoc practices into a more structured and documented approach that delivers process consistency in outcomes via repeatable processes centred around automation and tooling.
Although it is always tempting to dive straight into technical considerations, the benefits of administrative controls cannot be overlooked in secrets management. At a minimum, secrets management benefits from a formally agreed and documented policy that contains clear details of required standards as well as clear rules as to the requirements for securing API secrets. Although supported and implemented via tooling, it is important that all parties involved understand the required outcomes and how best to achieve them so that they are “bought in” to the process.
One key requirement is to define clearly what is meant by “secrets. It is especially important to differentiate between different grades or types of secret, as well as to make clear distinctions between the classification of secrets versus identifiers, and to determine the different requirements for each. Secrets commonly include authentication details such as passwords, connection strings and other information that authenticate access for users to API services, whereas system identifiers (such as hostnames and IP addresses) may still need to be confidential and shared selectively on a need-to-know basis even though they may be known to multiple parties and are not true secrets. Edge cases such as database connection strings may include elements of both, so clarity is required. The advantage of effective, well distributed and clearly communicated policies and standards means that the use of technical measures is more likely to be consistently observed by all employees.
These administrative controls should cover all phases of a secret’s lifecycle – we will look at each phase of this lifecycle in turn below.
The first step in the lifecycle of any secret is its generation or creation, and there are even at this early stage both best practice patterns to be followed as well as anti-patterns that may lead to security weaknesses which must be avoided.
Perhaps the most common mistake in secrets generation is to either create a secret that is valid in perpetuity or else to implement a system that does not evaluate whether a secret is currently valid. It is important that secrets have an associated validity beyond their binary existence: all mechanisms based on the use of secrets should evaluate both whether a secret has expired (reached the end of its lifetime as defined at time of creation) as well as whether it has been revoked (cancelled prior to its end-of-life date due to its being leaked or compromised, for example). Secrets should therefore be assigned a finite life, and subject to both rotation and minimum required lifetime. This means that a secret should not be valid for longer than is required, and once in use, a secret should ideally be changed periodically. When secrets stay in use for a long time, multiple people typically get access to it. The greater the number of people that have knowledge of a secret the greater the chance that they may be leaked by former employees or be hacked by malicious agents. In addition to their short-term rotation, most secrets will have an overall validity period also, after which any instance of it is no longer valid, regardless of rotation. Since a secret is generated for a specific purpose or partner it will generally have a naturally defined upper lifetime, such as to coincide with a partner’s contract term for example.
Another common issue is that secrets are set up to grant global access to an API, often providing access to all methods and functions, as well as to all data, and relying on client-side processes to filter and extract data that is required from the returned structure. However, this binary or “all or nothing” approach can often introduce security weaknesses, in returning more data than an API user should have access to. API keys should be assigned specific access permissions in exactly the same manner as is used in authentication for web application front-ends. This can incorporate both subject-object data mappings to determine which data belongs to a given user (and hence is accessible by them) as well as a form of Role-Based Access Control (RBAC) to define which API functions and methods a given secret such as an access token provides access to.
Although APIs can use a range of authentication mechanisms, many of them such as passwords and tokens can be of variable strength. The same best practices that apply to any password security (such as length, complexity, uniqueness, and discoverability/predictability requirements) can be applied to all secrets and credentials generated. It is generally advised to automate the creation of tokens, passwords, and other secret credentials for various reasons: their creation is not subject to variable quality by individual users, they are not subject to local storage or informal distribution and can be guaranteed to meet organisational strength requirements using agreed upon algorithms.
Secret sprawl refers to a security anti-pattern in organisations where secrets management is performed in an ad hoc manner. In such an environment, although each developer and other team member may well be making a best effort to manage secrets securely, they are not following a standardised practice but each following their own initiative, and secrets can become distributed and dispersed in various ways across different codebases, systems and platforms. This makes them hard to track in an organized inventory, causing issues such as difficulty ensuring keys are appropriately revoked or that all access credentials to a given system can be provably listed and audited.
Best practice involves ensuring that secrets are separate from application code, and ideally from other, non-secret, configuration data. The use of multiple cloud platforms has accelerated the development of dedicated secret management systems or “Secrets as a service” solutions such as Doppler and Hashicorp Vault, to store secrets securely and deploy them to a given environment. Where these solutions are not available, a database solution can offer some (though not all) of the same benefits: storing secrets within a database at least ensures typically that a formal integration is required (ensuring repeatability), that every secret is stored in a consistent manner, that secrets are stored with a consistent set of metadata each time, and that it is possible to easily determine access rights to each secret using built in database queries.
It is important not just to provide a secure method of storage for secrets, but to use technical measures to detect where secrets may have been stored outside of this system by employees not familiar with the correct processes or failing to follow them in certain instances. These technical measures can involve the scanning of code repositories such as Git to detect secrets embedded within code, and the use of “git add *” commands within version control systems to eliminate the introduction of hardcoded or embedded secrets. Some data leakage prevention (DLP) and detection solutions are also able to integrate with services such as enterprise cloud storage and mail services (MS Office 365) to detect credentials that have been shared via email or file share and are still remnant as artifacts in sent mail folders etc.
A final consideration for secrets storage is the encryption of the data at rest. This may take multiple forms, both ensuring that the dataset as a whole is encrypted at an application and filesystem level, but also where applicable that secrets are stored in the most secure form that is possible to support the given application – for example, passwords are typically not stored directly, but are hashed (using a per-password or per-customer salt) and only the hash itself recorded. This permits verification of passwords submitted during authentication, but without leaving the password susceptible to leak or theft from the server in case of partial compromise of the system.
Secrets distribution is often poorly implemented, especially if secrets are being shared with partner organisations and other third parties to provide access to partner APIs. Different organisations may struggle to agree a common approach to secrets distribution to each other if they do not have technical systems in common with one another.
At the very least, it is advised not to share secrets unencrypted (i.e., in plaintext) using methods such as email or instant messaging applications such as MS Teams or Slack. Not only is the password transmitted in plaintext but typically it will remain in the message history, logs, or “sent items” in plaintext also. Similarly, distribution of API credentials via a helpdesk ticketing system can leave secrets exposed to those who should not have access to them.
Secure distribution of secrets can often seem like a “chicken and egg” situation in which it is difficult to establish trust and proven identities in order to provably distribute keys securely to the correct individual. Public key cryptography generally offers one good approach and is often utilised within secrets management software.
Secrets such as API tokens can sometimes be implemented as a “binary” measure in that without a key, no access is possible to the API, but successful authentication with a credential provides unlimited access to all data via all API methods. This is especially common if an API is initially implemented from the standpoint of a “middleware” layer to support a GUI such as a web interface. Measures such as access control restrictions or record filtering can sometimes be implemented in these cases within the front-end (application) layer rather than at the API layer. When direct API access is then granted as an alternative access method at a later date, if the application is not rearchitected to move the logic to the API layer (or the controls alternatively not duplicated at the API layer) then direct API access can sometimes provide authenticated users with excessive access to data without appropriate restrictions.
Instead of granting global access, it is worth looking at the different API use cases for different API consumers and then establishing suitable authorization sets to certain methods and functions as needed using the principle of least privilege (granting each consumer of the API access only to the methods and functions they require and no more). This is normally achieved using some form of Role Based Access Control (RBAC) so that individual accounts and tokens are mapped to named roles, rather than having privileges directly assigned in an ad hoc manner.
Most APIs will be screened by at least a packet filtering firewall, if not an application layer firewall, and these are able to apply source restrictions so that only certain IP addresses can access the API. Although this can be done globally at the firewall level, it is more secure to map specific source IPs to specific tokens where possible. This is likely not possible for open public APIs, especially those that do not require registration, or which are designed for human access such as developer integration. However, in some use cases, such as inter-business (B2B) API usage accessed on a server-to-server basis only, it may be possible to identify corporate internet gateways and to apply access restriction. The advantage here is that even if an access token is compromised, an attacker may not be able to directly and immediately take advantage of it if their access source IP does not match that registered to the token.
Logging of significant events such as authentication attempts or unusual usage patterns (such as an access token or key being used sporadically from different source IPs) should be logged and, where appropriate, altering and monitoring put in place to recognise and respond to these security events. In spite of following all the best practices outlined in this blog post, a security incident relating to your API is still possible – most security controls reduce or mitigate risk rather than eliminate it altogether. If the worst happens, then it is important to at least be able to detect unauthorized access after it has occurred even if it could not be prevented. This allows teams to implement incident response procedures to limit the damage done and implement recovery procedures as needed.
The final tip we can offer regarding management of API secrets is to consider the risks where secrets are issued to third-parties outside your own organisation. As strong as your own security practices may be, it is difficult to ensure that all partners and vendors conform to best practices in using and managing secrets also, even if an onboarding and assurance programme is followed to perform due diligence.
It is also worth considering who or what an API key is being provided to or for – is the key or credential issued to a specific (named) individual within an organisation, or is it simply assigned an issued to an organisation such as a business partner? The latter case may present greater risk since the token may be known by and shared within a broad and untracked group of individuals, increasing the likelihood of token misuse or loss.
The final step in API secrets management is the revocation of a secret: that is, voiding the secret so that it is no longer active for its issued purpose. This can occur naturally at the end of the secret’s issued lifetime as it naturally expires if the API has been built to support and assign validity periods to secrets. It can also be necessary often to terminate or revoke a secret early on during its validity period if the token or secret has been found to be compromised. In such a case since it is no longer safe to use the secret (since it is known to an attacker), the secret must be put through a process of revocation by which it is actively marked as being null or unsupported for continued usage and rejected if offered by a client. It is important that secrets that are revoked are not simply deleted, but that they remain present and in a revoked state – this allows logging of attempts to continue to use the secret for example, preserves a valid transaction history for the secret, as well as permits restoration of the secret if it was subsequently found to have been revoked in error.
How can AppCheck help you in securing your APIs?
Native API Scanning Intelligence
AppCheck includes specific API scanning technology that is able to build up a map of a scanned API using WSDL and OpenAPI/Swagger files, seeded targets and similar technologies, to ensure that the API is fully modelled and scanned.
API Authentication
AppCheck supports API authentication methods including API access keys to ensure that APIs can be authenticated against, and all methods fully probed for vulnerabilities.
Custom Vulnerability Checks from First Principles
AppCheck includes API-specific vulnerability checks written by our in-house penetration testers, to detect previously unknown vulnerabilities from first principles.
AppCheck helps you with providing assurance in your entire organisation’s security footprint. AppCheck performs comprehensive checks for a massive range of web application vulnerabilities – including API security weaknesses – from first principle to detecting vulnerabilities in in-house application code.
The AppCheck Vulnerability Analysis Engine provides detailed rationale behind each finding including a custom narrative to explain the detection methodology, verbose technical detail, and proof of concept evidence through safe exploitation.
AppCheck is a software security vendor based in the UK, offering a leading security scanning platform that automates the discovery of security flaws within organisations websites, applications, network, and cloud infrastructure. AppCheck are authorized by the Common Vulnerabilities and Exposures (CVE) Program as a CVE Numbering Authority (CNA).
As always, if you require any more information on this topic or want to see what unexpected vulnerabilities AppCheck can pick up in your website and applications then please contact us: info@localhost
No software to download or install.
Contact us or call us 0113 887 8380
AppCheck is a software security vendor based in the UK, offering a leading security scanning platform that automates the discovery of security flaws within organisations websites, applications, network and cloud infrastructure. AppCheck are authorized by te Common Vulnerabilities and Exposures (CVE) Program aas a CVE Numbering Authority (CNA)
