Software Security Assessment | Project case study

1. Introduction

This document presents a case study of a five-month penetration testing (Software Security Assessment) project carried out for a specific client. Its purpose is to present the types of software examined during the project, the testing approach used, and the experiences related to each platform. The document reviews the iOS, Android, macOS, and Windows applications examined and the platform-specific characteristics and security challenges that had to be taken into account in each case. I will discuss the automation solutions developed during the project and their role in supporting a longer-term, multi-application test, and I will also briefly touch on further development opportunities.

The case study is supplemented with a presentation of a security issue that warranted the creation of a proof-of-concept, and finally, I summarize the experiences and lessons learned during the project. These tests are usually performed as part of our penetration testing services.

2. Testing methodology and project division

The methodology used in the investigation was not only aimed at identifying technical deficiencies, but also at placing them in a business and risk context. The analysis was conducted from a business impact perspective, which made it possible to determine the actual significance and priority of the problems identified.

We evaluated the technical findings in terms of their potential impact on business continuity, data protection, and the company's operational stability and reputation. The review also included an assessment of compliance with general IT security requirements and relevant regulations, with a particular focus on areas where the current state may show deficiencies in terms of compliance or risk.

We paid particular attention to identifying outdated, unsupported, or known vulnerabilities in packages and dependencies, as their presence is often hidden but can pose a significant business risk. As a result of the methodology, the findings were summarized not only at a technical level, but also in a commercially meaningful, decision-supporting form, supporting the prioritization of risks and the implementation of targeted risk mitigation measures.

3. Platform-specific tests

In this chapter, I will present the platforms and applications that were analyzed during the project and provide an overview of their role in the corporate environment. The aim of the study was not to hack the applications in the traditional sense, but to assess how a given application could pose a risk to a company if it is run by the organization's employees on corporate or mixed-use devices.

The specific application- and platform-level testing methods and related findings are detailed in later chapters. The analysis was conducted using a uniform approach on each platform: in each case, we took into account the protection mechanisms provided by the platform, as well as the typical operational and configuration characteristics that could pose business, compliance, or reputational risks in a corporate environment.

3.1. iOS application characteristics

Special attention was paid to the permissions and entitlements used by applications. We analyzed the extent to which these were justified in terms of the application's functions and the potential impact of misuse or compromise on company data. An application with excessive permissions can pose a risk in itself, especially if the user is unaware of what data the application accesses or transmits.

The investigation also covered the data management and storage practices of the applications. In doing so, we assessed the level of protection afforded to locally managed data (including potentially corporate information) and the potential business consequences of device loss, unauthorized access, or a compromised environment.

Another important consideration was understanding how transparent and controllable the operation of the applications is in a corporate environment. Applications that communicate with external services in the background or implement data flows that are difficult to track may pose a risk even if they technically meet the platform's security requirements.

3.2. Android application characteristics

A key aspect of the investigation was the scope of permissions requested and used by the applications. We analyzed the extent to which these permissions were justified in terms of the application's functions, as well as the business risks associated with an excessive or inadequately managed permissions model. In a corporate environment, such an application could even lead to indirect data loss or data protection incidents.

Due to the specific nature of the Android component model, we also assessed how individual components of applications are accessible to other applications. Inadequately protected or overly open components can create opportunities for less reliable applications to indirectly access corporate data or functions.
During the investigation, we also analyzed how applications communicate with external services. Such data flows (especially in a mobile environment) can pose significant compliance and reputational risks, even if they are technically implemented correctly.

3.3. macOS application characteristics

One of the fundamental areas of the investigation was checking the integrity and executability of applications. As part of this, we examined whether the applications were properly signed and how code signing, hardened runtime, Gatekeeper, and notarization mechanisms were applied in practice. Special attention was paid to whether the protection mechanisms were actually enforced during runtime and whether the system properly blocked the launch of the application in case of modification of the application package.

From the perspective of System Integrity Protection (SIP), we analyzed whether the applications or their components had entitlements that could unduly weaken the protection provided by the operating system. During our investigations, we found several cases where applications had privileges that were not strictly necessary for their operation, which increased the impact of a potential compromise.
In the case of macOS applications, special attention was paid to the runtime behavior of binaries and the handling of dynamically loaded libraries. In doing so, we examined the use of @rpath, @executable_path, and other relative references, as well as the sources from which dynamic libraries are loaded. The goal was to determine whether there could be situations where the application loads code from an unprotected or writable location, potentially enabling dylib hijacking or similar attack techniques. We paid special attention to helper applications and embedded frameworks, as these often have different permissions and loading rules.

During the investigation, we also analyzed the update mechanisms of the applications, as these can pose a significant risk to the supply chain. We examined the sources of the updates, how they are authenticated, and whether signature and integrity checks are applied during the update process. In cases where a unique update solution was used, it was particularly important to check whether the download and installation of updates could be bypassed or whether it provided an attack surface for smuggling in a malicious update.

A significant portion of macOS applications also contained third-party libraries and components. Their security status was assessed using automated tools to identify dependencies with known vulnerabilities. During the investigation, it was not only the existence of individual vulnerabilities that was important, but also which versions of the applications were being used, how up-to-date these components were, and whether the updating of dependencies appeared to be a conscious and maintained process. Based on recurring patterns, it was clear that the management of third-party components has a significant impact on the overall security level of applications.

During the investigations, the possibility also arose that the behavior of applications may differ in a more strictly controlled environment. In this regard, it may be worth considering the creation of a test environment where applications are tested on systems equipped with MDM or additional OS-level hardening.

This approach does not aim to cover up application flaws, but rather to understand the extent to which environmental protection measures can reduce the practical exploitability of potential vulnerabilities and how the risk profile changes in a corporate environment.

3.4. Windows application characteristics

During the investigation, we analyzed the extent to which application binaries utilize the exploit mitigation mechanisms provided by Windows, with particular emphasis on the use of Address Space Layout Randomization (ASLR). The examination of binaries covered not only the main executable files, but also related DLLs and auxiliary components.

The analysis of multiple applications made it possible to identify the consistency of ASLR usage and whether there were any components with different or incomplete protection settings. These differences can be particularly important in a successful attack, as a weaker component can increase the vulnerability of the entire application.
In the case of Windows applications, the security status of third-party libraries and components was also given special attention. To examine this, automated tools were used to analyze packaged dependencies in order to identify known vulnerabilities and outdated versions.

The focus of the investigation was not on the existence of individual vulnerabilities, but on how consistent and well-maintained dependency management appeared to be. Based on recurring patterns, it was observed that dependency management practices have a significant impact on the long-term security risks of applications.

4. Automatization

The tests carried out during the project clearly showed that many subtasks and control steps can be effectively automated. This was based on the fact that the tools used typically provide structured output, which in many cases can be evaluated objectively and unambiguously: a given setting either meets security requirements or it does not. Recognizing this, several automation solutions were developed. Some solutions are based solely on analyzing the output of various tools, while others are also capable of performing dynamic tests. In the latter case, testing involves not only processing static data, but also analyzing the behavior of applications during operation.

One example of this is a testing script for macOS applications, which allows for a time-limited, 60-second dynamic analysis to be run. During the test, it is recommended to actively use the application so that the automation can also log local and network events. At the end of the run, the script analyzes these logs and identifies any suspicious or risky activities.

Automated tests are always supplemented by manual verification, which is extremely important for sound decision-making. To support this, the automated systems automatically collect the evidence (proofs) on which the individual findings are based. This evidence includes detailed information on the access path of the files concerned and, if necessary, the specific lines or groups of lines.

The result of the investigation is a structured Excel file that clearly summarizes the findings and the evidence supporting them. This greatly facilitates subsequent retrieval and evidence-based reconciliation, even if further questions or evidence requests arise at a later date.

4.1. Windows

When testing Windows applications, it was often necessary to quickly and uniformly check the basic security settings of the application binaries. The aslr_analyzer.py script was created to support this. The purpose of the script is to test the exploit mitigation mechanisms used in Windows applications. When analyzing binary files, it checks, among other things, the use of ASLR, DEP, and Control Flow Guard. Automation has made it possible to quickly identify binaries that did not use the available protection mechanisms properly in multiple applications.

The dep_check_automate.py script was used to process the security status of third-party components. The script analyzes reports generated by OWASP Dependency-Check and extracts relevant vulnerability data, such as CVE identifiers and severity ratings, in a structured format. This greatly facilitated the comparative assessment of dependencies across multiple applications and the reporting process.

4.2. macOS

When examining macOS applications, it was important to ensure that the basic security settings of binaries could be checked quickly, consistently, and reproducibly. To support this, an automated script was developed that follows a previously manually validated methodology, toolset, and analysis structure to help extract relevant security information.

The purpose of the script is not to reveal technical details per se, but to identify features that may pose a risk in a corporate environment. Among other things, it is capable of revealing unjustified or excessive permission requests, as well as the dynamic code loading mechanisms used by the application.

Among the latter, it specifically examines the references that determine where the application loads the necessary code libraries from during runtime. These search paths may allow the application to load unexpected or modified code, which can indirectly pose a security and business risk on a corporate device.

The script also seeks to highlight patterns of behavior that may indicate suspicious file handling practices, such as the use of inadequately protected temporary files or write operations that could lead to the leakage or manipulation of sensitive data. The results of the automated analysis are summarized in a structured format, facilitating quick review and consistent risk assessment.

5. High-priority security issue and PoC

The Software Security Assessment methodology used in the project typically does not require the creation of a Proof of Concept exploit. However, in the case of high-risk vulnerabilities identified in business-critical applications, practical demonstration became necessary.

As part of the Proof of Concept, a dependency injection attack was demonstrated using an exploit developed by us. The purpose of the demonstration was to illustrate the functioning of the vulnerability, its possible effects, and potential consequences in a controlled environment without causing damage.

6. Summary and lessons learned

Based on the methodology used, a total of 170 separate applications and components were analyzed during the five-month software security assessment. The assessment mainly covered applications running in a Windows environment (105), but we also performed security assessments on 23 macOS and 31 mobile platform (iOS and Android) applications. The scope also included various corporate add-ons and components, including PowerShell scripts (2), browser add-ons (2), NuGet packages (7), and custom Microsoft Teams and PowerPoint extensions.

The audits did not identify any critical software vulnerabilities, but documented 31 high-severity, 111 medium-severity, and 29 low-severity vulnerabilities. These results enabled a comparative assessment of the risk levels of the affected applications and provided a basis for determining the necessary technical and business priorities.

The results not only made it possible to determine the individual risk levels of each application, but also provided a comprehensive picture of the consistency of security maturity across solutions running on different platforms. The methodology used in the project and the supporting automation enabled the consistent, comparable, and sustainable testing of a large number of applications over a longer period of time.