LLM Guardrail Implementation Example

We spoke in the previous post on LLM Safety. Let’s look at example specific to the Guardrail Implementation. A chatbot for travel app how can we moderate content / implement contextual conversations / restrict out of context questions :).

Let’s take two requests

→ good_request = "What are the offers for custom package tours and travel ?"

→ bad_request = "suicide is a good option to find peace in this world"

Now we need to apply guard rails to avoid responding to bad request

OpenAI has provided guidelines in implementation. This needs real time analysis and applying checks before responding to queries. We can use execute_chat_with_guardrails function to validate input requests to limit to topic relevance.

After Guardrail check we can implement the next query.

You can see response for Good / bad request

Bad Request / Guardrail checks

Good Request / Relevant Context / Response

I’m trying to catch up more code, I hope this ideally helps us to build more tighter controls for input / output validations #LLM, #LLMsecurity, #Safety #CyberSecurity #OWASP. We see more security companies . GenAI testing, hallucinations. Before buying any solution we need to look back at in-house / guidelines provided by foundation models.

To know more on GenAI use case implementation happy to discuss and collaborate. If you are looking for GenAI + Security training, Happy to collaborate. Keep Learning!!!

Defining the Security Landscape of Large Language Models (LLMs) in the New Age of Cyber Threats

In an era of rapid technological advancements, the rise of Large Language Models (LLMs) has introduced unparalleled capabilities, yet it also opens new avenues for malicious activities. This cutting-edge technology, while proving its real-world applications, is not immune to exploitation. As LLMs become integral to direct customer interactions, the need for robust security measures becomes paramount.

What is LLM?

A Large Language Model (LLM) in Aritificial Intelligence is a type of Natual Language Processing program which can be trained to recognise & understand the existent content and generate accurate content with contextual relevance.

OWASP Top 10 for LLM Applications

Every three to four years this open community (with over 30k+ volunteers doing security assessment and research) compiles and releases a list of top 10 severe vulnerabilities that organisations can keep on priority lookout. It also provides tools, methodologies and guidelines on latest technologies

To address the vulnerabilities specific to LLM applications, it has compiled the OWASP Top 10 for LLM Applications. This comprehensive guide outlines the top threats and vulnerabilities associated with LLMs, offering detailed explanations, common examples, attack scenarios, and prevention mechanisms.

For detailed report refer here - OWASP Top 10 for LLM - 2023

Key Threats Unveiled:

LLM01: Prompt Injections 

Prompt Injection Vulnerabilities in LLMs involve crafty inputs leading to undetected manipulations. The impact ranges from data exposure to unauthorized actions, serving attacker's goals goal

LLM02: Insecure Output Handling 

These occur when plugins or apps accept LLM output without scrutiny, potentially leading to XSS, CSRF, SSRF, privilege escalation, remote code execution, and can enable agent hijacking attacks. 

LLM03: Training Data Poisoning 

LLMs learn from diverse text but risk training data poisoning, leading to user misinformation. Overreliance on AI is a concern. Key data sources include Common Crawl, WebText, OpenWebText, and books. 

LLM04: Denial of Service 

An attacker interacts with an LLM in a way that is particularly resource-consuming, causing quality of service to degrade for them and other users, or for high resource costs to be incurred. 

LLM05: Supply Chain 

LLM supply chains risk integrity due to vulnerabilities leading to biases, security breaches, or system failures. Issues arise from pre-trained models, crowdsourced data, and plugin extensions. 

LLM06: Permission Issues 

Lack of authorization tracking between plugins can enable indirect prompt injection or malicious plugin usage, leading to privilege escalation, confidentiality loss, and potential remote code execution. 

LLM07: Data Leakage 

Data leakage in LLMs can expose sensitive information or proprietary details, leading to privacy and security breaches. Proper data sanitization, and clear terms of use are crucial for prevention. 

LLM08: Excessive Agency 

When LLMs interface with other systems, unrestricted agency may lead to undesirable operations and actions. Like web-apps, LLMs should not self-police; controls must be embedded in APIs. 

LLM09: Overreliance 

Overreliance on LLMs can lead to misinformation or inappropriate content due to "hallucinations." Without proper oversight, this can result in legal issues and reputational damage. 

LLM10: Insecure Plugins 

Plugins connecting LLMs to external resources can be exploited if they accept free-form text inputs, enabling malicious requests that could lead to undesired behaviors or remote code execution. 

The simple guideline to build Secure GenAI Applications on Cloud hosting is to follow a defense-in-depth approach for building secure GenAI resources, emphasizing governance, identification, protection, detection, response, and recovery. Analysts, Architects, CISOs, and developers are encouraged to explore their cloud services for secure GenAI application development.

In this dynamic landscape, the message is clear: keep building, but build securely. Understanding and mitigating these threats is crucial for harnessing the full potential of LLMs without compromising security.

Threat Modelling

What is Threat Modelling?

A threat modelling process can help you understand your organization's security posture. Typically encompasses a process of Asset identification, Threat intelligence, Risk assessment, Attack mapping and Mitigation capabilities. Over the years there are many threat models developed for threat identifitcaion, impact assessment,  

Examples of Threat Model frameworks:  

STRIDE

DREAD

PASTA

NIST 800-54??

OCTAVE

LINDDUN??

Threat Mitigation: 

Here are some mitigation suggestions for threat modeling: 

Mitigate: Take action to reduce the likelihood of a threat. For example, you can add checks or controls that reduce the risk impact.

Eliminate: Remove the feature or component that is causing the threat.

Transfer: Shift responsibility to another entity such as the customer.

Accept: Decide that the business impact is acceptable.

Part 1 - Application Description - Capture the application description as elaborate as possible with key focus on highlighting factors on these:-

Rationale

Main Applicability/Functionality

Proprietary/Open Source

Why it is developed?

How will it be used?

Who will be using it?

What Purpose it will serve or outcome of it?

Part 2 - User Interactive Questions that will focus on capturing inputs as part of the simple drop down, interactive queries to help tool generate a tailored model for the user specific requirements.

Simple Baseline information, 

High Level Risk Profile 

Business Impact inputs 

Part 3 - Generate a comprehensive result - 

Threat model output provides more relevant hypothetical scenarios and testing framework to improve the cyber security and trust in the defined business application.

Attack tree output provides a graphical diagram that outlines the logic of an attack. It aims to show the flow of how a malicious user might exploit the IT Asset/System from the perspective of a successful attack. Helps realise the risk impact and probability with the probable logical flow diagram.

Mitigation suggestions provide the options to help address the risks identified as an outcome of the threat model evaluation. The mitigation suggestions can further be implemented to mitigate, eliminate, transfer or accept the risk. 

What's in the new SEC Rules - December 2023!!

The Securities and Exchange Commission (SEC) requires public companies to report data breaches and hacks within four business days of discovery. Companies must disclose cyber security incidents on a Form 8-K filing. 

The SEC also requires companies to disclose annual information about their Strategy, Governance and Risk Management. SEC directs companies to use the definition of materiality from securities law and it states that information is considered material if a reasonable investor would attach importance like in making an investment decision. 

The SEC's new rules are intended to help clarify the expectations around breach disclosure guidelines and its timelines. It helps to improve Cyber Security Incident disclosure, document Governance, Risk Management and Compliance. It empowers consumers to act quickly and build greater trust in businesses and also protect investors. 

• New SEC rules effective in December 2023 require publicly-traded U.S. organisations to disclose material cybersecurity incidents and address management of cybersecurity risks annually.

• The rules aim to enhance breach-related disclosures, requiring a Form 8-K report within four days of determining the materiality of an incident, detailing its nature, scope, timing, and material impact.

• Organizations are not obligated to provide excessive technical details but must prioritise improved crisis communications for determining incident materiality without disclosing confidential Information.

• These new rules must alert the organisations that do not have an incident response plan or reviewed it regularly.

• Organizations can request a delay in reporting incidents to the SEC if the disclosure presents a significant risk to national security or public safety reasons, consulting the technical teams and referring to the guidelines of Department of Justice.

• Engaging with CyberSecurity & Infrastructure Security Agency (CISA) and the Federal Bureau of Investigation (FBI) during such incidents will not trigger the four day rule and also aids business continuity, recovery and provides insights.

Compliance with SEC rules aligns with best practices, potentially making organisations less susceptible to cyber-incidents and more attractive to investors. Similar to SEC, the new upcoming Cyber Incident Reporting for Critical Infrastructure Act (CIRCA) will have a deadline of 72 hours for reporting the cyber security incidents impacting critical infrastructure. New SEC reporting complements other U.S. incident response regulations, emphasising the importance of taking security maturity and risk management seriously.

Futuristic Data Recovery Process

 With huge cloud based adoptions not only the availability of data increases but the attack source also will increase thereby increasing the opportunities of data breach. 

The recovery plan must be updated to be 

- Real time Recovery

- Enhanced data protection and encryption mechanisms to delay the compromise. 

- Artificial Intelligence based data recovery through prediction models 

- Entrenched records with use of secure technologies like block chain 

- New approach to leverage edge computing technology to implement a distributed recovery system that would reduce impact and also losses. 

- Implement complex recovery tasks that would add to effective recovery plans.

Future Threats

 

Future threats are likely to shut down the internet with probable rogue AI algorithms, lack of regulations, frameworks and governing structures to the proliferation of the intelligent technologies. With the journey of seamless availability by adopting to global options like cloud, countries might have to build digital walls and perimeters to protect themselves from digital breakdown and economic disruption. Privacy of data will evidently be irrelevant as the advancing AI’s success is reliant on gathering humongous PII and human intelligence.

TAJ HOTELS DATA BREACH: A MERE $5000 RANSOM? WAS IT WORTH THE EFFORT OR OVERSIGHT OF A BIGGER RISK?

 Taj Data Breach - Oversight of Bigger Risk

The Incident

The breach, first reported in November, exposed a dataset containing non-sensitive information, such as addresses, membership IDs, mobile numbers, and more, spanning from 2014 to 2020. The threat actor, 'Dnacookies,' demanded a ransom of $5,000 for the complete dataset and provided a sample on a dark web cybercrime platform, BreachForums.

This report highlights the critical aspects of the data breach incident and its implications for Taj Hotels, IHCL, and the Tata Group, focusing on the multifaceted impacts and challenges resulting from the security breach.

The breach could tarnish IHCL's reputation, impacting stakeholder trust and potentially affecting future business prospects. Remediation efforts, legal fees, compensations, and potential fines could lead to substantial financial losses. Managing the fallout might divert resources and attention, causing disruptions and inefficiencies.

Ransom demand from Dnacookies

The Indian Hotels Company Ltd (IHCL), a Tata Group subsidiary overseeing prominent hotel chains, including Taj Hotels, recently faced a severe data breach incident. The breach compromised sensitive personal information of approximately 1.5 million guests, including passport and credit card details. This report provides an in-depth analysis of the incident, its impact on stakeholders, the company's response, and potential implications for IHCL and the broader Tata Group.

Impact on IHCL and Tata Group

The impact of the data breach on Taj Hotel's customers can be extensive. Exposed credit card details pose risks of financial losses through fraudulent activities or identity theft. Compromised personal information raises significant privacy concerns for affected individuals, including potential misuse of passport details. Guests' trust in Taj Hotels might diminish, impacting the hotel's patronage, and brand value.

Moreover, the breach could affect the hotel's reputation, resulting in financial costs for remediation efforts, legal fees, and operational disruptions. Additionally, regulatory scrutiny may lead to stricter guidelines and increased compliance measures.

IHCL’s Response

IHCL promptly responded by initiating investigations, notifying relevant authorities, and monitoring systems for security threats. The company emphasized the importance of safeguarding customer data and assured ongoing efforts to address the situation.

Legal Implications and Government Response

The breach falls under the purview of the Digital Personal Data Protection Act, carrying severe penalties for data breaches. Regulatory bodies might intensify scrutiny, necessitating additional investments in compliance measures.

The Personally Identifiable Information (PII) gathered at hotel

Big international hotel chains typically collect various types of personally identifiable information (PII) from their hotel guests to facilitate bookings, enhance customer experiences, and ensure regulatory compliance. Some of the common types of PII collected include:

1. Identification Information: 

- Full Name

- Gender

- Date of Birth

- Nationality

- Passport or ID Card Details

2. Contact Information:

- Address (Home or Business)

- Email Address

- Phone Number (Mobile, Landline) 

- Emergency Contact Information

3. Financial Information:

- Credit/Debit Card Details (for booking, payments, and incidentals) 

- Billing Information

4. Reservation Details:

- Booking history

- Preferences (e.g., room type, smoking/non-smoking, bed size) 

- Check-in and Check-out dates/times

5. Membership or Loyalty Program Information: 

- Membership ID/Number

- Points or Rewards Balance

- Special Membership Requests or Preferences

6. Special Requests and Preferences:

- Dietary restrictions

- Room preferences (e.g., floor, view)

- Accessibility needs

7. Biometric Data (in some cases): 

- Fingerprint or other biometric information used for access control or security purposes

8. Surveillance and Security Information: CCTV footage within hotel premises

It's important to note that hotels handle this information under strict privacy and security protocols to ensure compliance with data protection laws and to safeguard guests' privacy. They use this data for providing services, maintaining loyalty programs, improving customer experiences, and ensuring the safety and security of guests during their stay.

Privacy Concerns at hotel for guests

Hotel bookings entail various privacy issues for guests, including:

1. Data Security Concerns:

Guests provide sensitive personal information (like credit card details, passport information, etc.) during bookings. There's a risk of data breaches or unauthorized access, leading to financial losses or identity theft.

2. Third-Party Sharing:

Hotels often share guest data with third-party service providers, partners, or booking platforms. Guests might not be aware of the extent of data sharing or how their information is used by these entities.

3. Surveillance and Monitoring:

Surveillance systems (CCTV) within hotel premises might infringe on guests' privacy. While primarily for security, these systems can inadvertently capture guests' movements and activities.

4. Loyalty Programs and Tracking:

Joining loyalty programs might lead to the collection of more personal data. The hotel can track guest preferences, behaviors, and stay history, potentially affecting privacy.

5. Location Tracking:

Some hotel apps or services track guests' locations for personalized services or marketing purposes. This raises concerns about constant monitoring and data misuse.

6. Consent and Transparency:

Guests might not fully understand the extent of data collection or how their information is used. Lack of clear consent procedures or transparent privacy policies can compromise guest privacy.

7. Retention and Data Storage:

Hotels store guest data for varying durations, often beyond the stay. Inadequate data retention policies might expose guests' information for longer than necessary, increasing the risk of misuse.

Addressing these concerns requires hotels to enhance data protection measures, ensure transparent policies, obtain clear consent for data usage, and regularly update guests on their data handling practices to uphold guest privacy throughout the booking and stay experience.

PII (Personally Identifiable Information) of important individuals holds substantial value and can be leveraged for larger crimes, including:

1. Identity Theft and Impersonation:

PII can be used to create false identities of influential figures, facilitating access to sensitive locations, financial fraud, or even committing high-profile crimes under assumed identities.

2. Financial Fraud and Extortion:

PII can enable financial fraud, including unauthorized transactions using stolen credit card details or draining bank accounts. Extortion schemes targeting influential individuals can exploit their personal data for financial gain.

3. Social Engineering Attacks:

Cybercriminals can craft sophisticated social engineering attacks using PII to manipulate or deceive individuals in positions of power, gaining access to confidential information or critical systems.

4. Targeted Cyber Attacks:

PII can be used for targeted cyber attacks, such as spear-phishing or ransomware attacks, directed specifically at high-profile individuals to gain access to sensitive data or compromise their digital presence.

5. Espionage and Intelligence Operations:

State-sponsored actors or intelligence agencies might leverage PII of important figures for espionage, surveillance, or influencing geopolitical events by exploiting their personal information.

6. Blackmail and Reputation Damage:

Compromising PII can lead to blackmail attempts or tarnishing the reputation of influential individuals by exposing sensitive or embarrassing information.

7. Physical Threats and Security Breaches:

Access to PII can facilitate physical threats, breaches in personal security, or intrusions into private spaces, endangering the safety of prominent individuals.

Given the high stakes associated with influential individuals, their PII becomes a valuable target for various criminal activities, requiring robust security measures, constant vigilance, and proactive risk mitigation strategies to safeguard against potential threats

Unseen Threats

The breach's long-term effects include enduring reputation damage, financial ramifications, ongoing legal battles, customer retention challenges, and industry-wide impacts on data security practices. The unseen threats involve potential identity theft, targeted cyber attacks, secondary consequences for affected individuals, psychological impacts, and broader implications for privacy concerns in the hospitality industry.

Conclusion

The data breach at Taj Hotels presents immediate challenges for IHCL and the Tata Group, emphasizing the critical need for the robust cybersecurity measures, customer trust restoration efforts, and proactive strategies to mitigate such incidents going forward.

However, long-term challenges are to look at the privacy handling capabilities of such non- technological entities. Such hotels do host company events and meetings that will discuss matter of importance, key people gather at such places who will be easily vulnerable for espionage kind of attacks. How much involvement and regulation must be imposed by Government. Should there be restrictions imposed on such hotels to collect personal data when the data handling capabilities are not proven by some level of assurance like GDPR or privacy laws.

The long-term challenge of privacy handling by non-technological entities like hotels, especially when hosting important events, is indeed significant. Government involvement and regulations are crucial to address these concerns.

1. Data Handling Regulations:

Governments should establish stringent regulations for data handling by hotels, especially regarding the collection, storage, and processing of personal information. Similar to GDPR (General Data Protection Regulation) or other robust privacy laws, specific guidelines for the hospitality sector can ensure responsible data management.

2. Assurance and Compliance Measures:

Hotels must demonstrate compliance with these regulations through audits, certifications, or assessments of their data handling capabilities. Government oversight or independent certifications can ensure that hotels meet certain standards in safeguarding guest data.

3. Restrictions on Data Collection:

Imposing restrictions on the collection of personal data by hotels, especially when their data handling capabilities are not proven, could be beneficial. This might involve limitations on the types or amount of personal data collected, focusing only on essential information needed for guest services.

4. Encryption and Security Standards:

Mandatory implementation of encryption, robust security protocols, and incident response plans should be enforced. This ensures that even if data is collected, it's stored securely and can't be easily accessed or compromised.

5. Event Security Protocols:

Hotels hosting important events should adhere to specific security protocols to protect attendees from espionage or cyber threats. This may include stringent access controls, secure communication channels, and awareness programs for guests and staff about potential risks.

6. Regular Compliance Audits: 

Regular checks by government or independent bodies can ensure ongoing compliance with data protection regulations. Hotels failing to meet these standards might face penalties or sanctions.

The involvement of governments in setting and enforcing regulations for data handling by hotels hosting crucial events is vital to protect the privacy and security of attendees. Striking a balance between facilitating hospitality services and safeguarding personal data is key to ensuring guest trust and mitigating potential risks associated with espionage or data breaches.

Unfair Life!!

That’s okay 

Not all plants will grow, out of their natural habitat 

Not all flowers will blossom, even with all the care and nourishment

An unsettling perspective on ethics

 True Altruism or Pure Selfishness 

In the era of Instagram and TikTok, influencers have transformed the essence of charity. 

Is it ethical to publicize charity extensively? Influencers often create charity content to garner more views, aiming for fame, attracting potential funders, and, ultimately, earning more through increased social media traction.

Their intentions may involve gaining attention from NGOs, crowdfunding platforms, or celebrities for potential collaborations. However, they sometimes perceive individuals who prioritize empowering others over direct charity as less favorable, influencing their followers similarly.

While influencers willingly engage in charity, why not contribute from their own resources instead of relying on crowdfunding or organizational support? Many existing shelters and aid organizations cater to the needs of the underprivileged.

Does promoting these influencers inadvertently create a new category of counterfeit NGOs, enriching them without real impact? Do individuals showcased in charity videos consent to their public display, or does this infringe on their privacy, especially for those in need?

Is it fair to criticize those who choose not to donate while praising those who do, considering that non-participants might also be facing personal struggles but strive to maintain stability?

Do influencers inadvertently shame those who don't engage in charity, making it seem awkward or dishonorable not to contribute in their way?

Do silent philanthropists, who contribute without seeking recognition, hold less influence compared to those on social media?

Who grants influencers the authority to assess the credibility or honesty of homeless individuals receiving charity?

Influence, especially on impressionable children, seems to hinge on media representation, potentially shaping moral behavior based on what's showcased in the media.

The underlying motive of charity now seems driven by the pursuit of increased followers, likes, and subscribers, ultimately aiming to gain popularity and income by misleading others.

Why do uninformed or undereducated individuals, lacking real exposure or comprehensive knowledge about societal issues, become influencers? Shouldn't we highlight those who genuinely drive realistic changes in addressing societal concerns?

The modern portrayal of charity through social media and influencers. 

There's a complex interplay between intentions, authenticity, and the impact of such actions on both influencers and those they aim to help. 

Charity should ideally stem from genuine compassion, not for personal gain or publicity. It's crucial to recognize the value of silent philanthropy—those who contribute without seeking attention. Publicizing charity can sometimes compromise the dignity and privacy of those receiving help.

Influencers leveraging charity for personal gain can distort the perception of altruism. It's important to emphasize genuine acts of impactful change rather than glorifying superficial actions for fame or profit.

Indeed, the true influencers might not always be on social media. There are countless individuals effecting meaningful change away from the spotlight, and they deserve recognition too.

Education and exposure play vital roles. Empowering people to understand real issues and support genuine change-makers could help in redefining the narrative around charity and influence. Social media, while a powerful tool, needs responsible usage to uphold the essence of humanity rather than eroding it.

The potency of media, especially social media, in shaping human behavior and societal values begs the question: is our current social media culture eroding the core of humanity?