Vibe coding is popular, but how good does “vibe security” compare to throwing traditional SAST tools at your code? “Vibe security review” seems to be a valuable addition to the aresenal here, and performs better than both Sonarqube and Bandit!
Here’s an intentionally poorly programmed Python file (generated by Le Chat with instructions to create a vulnerable and poorly coded text adventure game):
import random
import os
class Player:
def __init__(self, name):
self.name = name
self.hp = 100
self.inventory = []
def add_item(self, item):
self.inventory.append(item)
def main():
player_name = input("Enter your name: ")
password = "s3Lsnqaj"
os.system("echo " + player_name)
player = Player(player_name)
print(f"Welcome, {player_name}, to the Adventure Game!")
rooms = {
1: {"description": "You are in a dark room. There is a door to the north.", "exits": {"north": 2}},
2: {"description": "You are in a room with a treasure chest. There are doors to the south and east.", "exits": {"south": 1, "east": 3}},
3: {"description": "You are in a room with a sleeping dragon! There is a door to the west.", "exits": {"west": 2}},
}
current_room = 1
while True:
room = rooms[current_room]
print(room["description"])
if current_room == 3:
action = input("Do you want to 'fight' the dragon or 'flee'? ").strip().lower()
if action == "fight":
if random.randint(0, 1):
print("You defeated the dragon and found the treasure! You win!")
else:
print("The dragon defeated you. Game over!")
break
elif action == "flee":
current_room = 2
continue
command = input("Enter a command (go [direction], get [item]): ").strip().lower()
if command.startswith("go "):
direction = command.split("go ")[1]
if direction in room["exits"]:
current_room = room["exits"][direction]
else:
print("You can't go that way.")
elif command.startswith("get "):
item = command.split("get ")[1]
player.add_item(item)
print(f"You picked up {item}.")
else:
print("Invalid command.")
if __name__ == "__main__":
main()
The code is made unnecessarily complex, and allows user input to be injected directly into the command line. It also has a hardcoded “password” that is not really used for anything. Let’s check the security of this with 3 methods:
- Sonarqube Cloud (free version)
- Bandit (Python SAST tool)
- Github Copilot with GPT-o4
Sonarqube is run with the default ruleset “Sonar Way” with no extra configuration effort. The scan detects “2 security hotspots” and some maintainability issues. A security hotspot is a potential weakness that the SAST tool can’t conclude on, and will require manual review.
The first hotspot is the “password”:
Sonarqube also gives guidance on how to assess if this is dangerous, and how to fix it, which is quite helpful. The other hotspot is “weak cryptography” when the game is using random.randint to choose whether fight or flee is the winning strategy when encountering a dragon – not a (software) security issue. Dragons can be dangerous, though!
Now, let’s try Bandit.
Badit did its job – and provides links to more information, including CWE references. Blue is “not good” and red is “bad”.
OK, now for Github Copilot.
Prompt – and then followed by the formatted report!
You are a professional software quality expert and security test engineer. Please review this code and provide an overview of security weaknesses with references to CWE's, and suggested fixes. Analyse the current file. Create the output as a security vulnerability report in markdown format.
Security Vulnerability Report for spaghetti.py
This report provides an analysis of the security weaknesses in the provided spaghetti.py file. Each vulnerability is referenced with its corresponding Common Weakness Enumeration (CWE) and includes suggested fixes.
Vulnerabilities
1. Hardcoded Password
- Description: The password is hardcoded in the file (
password = "s3Lsnqaj"), making it vulnerable to exposure if the file is shared or accessed by unauthorized users. - CWE Reference: CWE-798: Use of Hard-coded Credentials
- Impact: Hardcoded passwords can lead to unauthorized access to sensitive resources.
- Suggested Fix:
- Store the password securely using environment variables or a secrets management tool.
- Example:
import os password = os.getenv("GAME_PASSWORD") if not password: raise ValueError("Password not set in environment variables")
2. Command Injection
- Description: The
os.systemcall uses unsanitized user input (os.system("echo " + player_name)), which can lead to command injection if a malicious user provides crafted input. - CWE Reference: CWE-78: Improper Neutralization of Special Elements used in an OS Command (‘OS Command Injection’)
- Impact: A malicious user could execute arbitrary system commands, potentially compromising the host system.
- Suggested Fix:
- Use safer alternatives like
subprocess.runwith proper input sanitization. - Example:
import subprocess subprocess.run(["echo", player_name], check=True)
- Use safer alternatives like
3. Unvalidated User Input
- Description: User input (
player_name,action,command) is not validated or sanitized, which can lead to unexpected behavior or vulnerabilities. - CWE Reference: CWE-20: Improper Input Validation
- Impact: Unvalidated input can lead to crashes, security vulnerabilities, or incorrect program behavior.
- Suggested Fix:
- Validate and sanitize all user inputs.
- Example:
player_name = input("Enter your name: ").strip() if not player_name.isalnum() or len(player_name) > 50: print("Invalid name. Please enter a valid name.") return
4. Predictable Randomness
- Description: The use of
random.randint(0, 1)for critical game logic (e.g., dragon fight outcome) is predictable and not cryptographically secure. - CWE Reference: CWE-338: Use of Cryptographically Weak Pseudo-Random Number Generator (PRNG)
- Impact: Predictable randomness can lead to unfair or exploitable outcomes in the game.
- Suggested Fix:
- Use a cryptographically secure random number generator if randomness is critical.
- Example:
import secrets if secrets.choice([True, False]): print("You defeated the dragon and found the treasure! You win!") else: print("The dragon defeated you. Game over!")
5. No Input Length Restriction
- Description: The
player_nameinput does not restrict the length, which could lead to buffer overflow or other issues in certain scenarios. - CWE Reference: CWE-1283: Improper Validation of Specified Quantity in Input
- Impact: Excessively long input could cause performance issues or crashes.
- Suggested Fix:
- Limit the length of user input and validate it.
- Example:
player_name = input("Enter your name: ").strip() if len(player_name) > 50: print("Name too long. Please enter a shorter name.") return
6. No Error Handling
- Description: The code lacks error handling for invalid commands, missing keys in dictionaries, or unexpected input.
- CWE Reference: CWE-703: Improper Check or Handling of Exceptional Conditions
- Impact: The program may crash or behave unpredictably when encountering errors.
- Suggested Fix:
- Add proper exception handling to ensure the program handles errors gracefully.
- Example:
try: direction = command.split("go ")[1] if direction in room["exits"]: current_room = room["exits"][direction] else: print("You can't go that way.") except KeyError: print("Invalid room configuration.") except Exception as e: print(f"An unexpected error occurred: {e}")
Summary
The code contains several vulnerabilities that could compromise its security and reliability. Addressing these issues will improve the robustness and safety of the application.
Recommendations
- Replace hardcoded sensitive information with secure storage methods.
- Avoid using
os.systemwith unsanitized input; use safer alternatives likesubprocess.run. - Validate all user inputs and restrict their length to prevent unexpected behavior.
- Use proper error handling to ensure the program does not crash due to invalid input or unexpected conditions.
- Consider using cryptographically secure randomness for critical game logic if necessary.
By implementing these fixes, the code will be more secure and maintainable.
Summary
Using Copilot to do a security review and generate a report created the most thorough assessment, which is not surprising. Bandit did quite well and detected the actually dangerous injection vulnerability. Sonarqube provides a nice UI but did not detect the one actually serious vulnerability here.
safecontrols 