Comprehensive Guide to Developing Cryptography CTF Challenges

Introduction

Cryptography challenges are a cornerstone of CTF competitions, testing participants' understanding of encryption, cryptanalysis, and secure systems. Well-designed challenges educate while entertaining, building skills that transfer directly to real-world security work.

Goals of This Guide

• Help you create challenges that are educational and fair
• Provide frameworks for different difficulty levels
• Offer practical examples and anti-patterns to avoid
• Enable you to write comprehensive writeups
• Ensure your challenges are properly balanced and tested

Challenge Design Principles

1. Educational Value

Every challenge should teach participants something useful about cryptography:

• Real cryptographic concepts and algorithms
• Common implementation mistakes
• Cryptanalysis techniques
• Security best practices (by showing what happens when they're violated)

2. Clear Intent

Players should be able to identify what type of challenge they're solving:

• Challenge title hints at the category (e.g., "Broken RSA", "Weak Random")
• Description provides necessary context without being a tutorial
• Required tools/knowledge are reasonable for the difficulty level

3. Fair and Solvable

• All necessary information is provided or easily discoverable
• No "guess what I'm thinking" moments
• Intended solution is feasible within competition timeframe
• Unintended solutions are either acceptable or prevented
• Difficulty matches the points awarded

4. Realistic Scenarios

Base challenges on real-world vulnerabilities when possible:

• RSA with small primes (real attack: factorization)
• ECB mode misuse (real attack: ECB penguin)
• Weak random number generation (real attack: predictable tokens)
• Padding oracle attacks (real attack: Lucky Thirteen)

Difficulty Levels

Beginner (100-150 pts)

Target Audience: Players with basic programming knowledge, minimal crypto experience

Concepts:

• Classical ciphers (Caesar, Vigenère, substitution)
• Simple encoding schemes (Base64, hex, ASCII)
• Basic XOR operations
• Recognizing common hash algorithms

Solution Path: Should be solvable with online tools or simple scripts (5-15 minutes)

Example: "Decode this Base64 string, then apply ROT13"

Easy (200-250 pts)

Target Audience: Players familiar with basic crypto concepts

Concepts:

• Modern ciphers with obvious weaknesses
• Frequency analysis
• Simple RSA scenarios (small e, common modulus)
• Basic hash collisions (MD5, deprecated algorithms)

Solution Path: Requires basic scripting or standard tools (15-45 minutes)

Medium (300-400 pts)

Target Audience: Players with solid crypto fundamentals

Concepts:

• RSA attacks (Wiener's attack, Coppersmith's theorem)
• Block cipher mode weaknesses (ECB, CBC bit flipping)
• Elliptic curve cryptography basics
• Stream cipher vulnerabilities (nonce reuse)
• Simple padding oracle attacks

Solution Path: Requires understanding of attack techniques and custom scripting (45-90 minutes)

Hard (500-700 pts)

Target Audience: Experienced crypto players

Concepts:

• Advanced RSA attacks (Boneh-Durfee, LSB oracle)
• ECC attacks (invalid curve, small subgroup)
• Complex oracle attacks (timing, error message leaks)
• Side-channel analysis
• Hash function attacks (length extension, collision finding)

Solution Path: Requires deep understanding and potentially hours of work (2-6 hours)

Expert/Insane (800-1000+ pts)

Target Audience: Elite players with extensive crypto knowledge

Concepts:

• Lattice-based cryptanalysis (LLL, BKZ)
• Advanced number theory attacks
• Zero-knowledge proof systems
• Post-quantum cryptography
• Novel attack combinations

Solution Path: May require research, complex mathematics, and significant computation (6+ hours)

Challenge Categories

1. Classical Cryptography

Good for: Beginner challenges, pattern recognition

Common Types:

• Caesar cipher with various shifts
• Vigenère cipher with known/unknown key length
• Substitution ciphers (simple, homophonic)
• Transposition ciphers (rail fence, columnar)
• Playfair, Beaufort, or Autokey ciphers

Design Tips:

• Provide sufficient ciphertext for statistical analysis
• Consider adding obfuscation layers (encoding before encryption)
• Hint at the cipher type in the challenge title or description

2. Symmetric Cryptography

Good for: All difficulty levels

Common Types:

• Mode Attacks: ECB reordering, CBC bit-flipping, CTR nonce reuse
• Padding Attacks: Oracle attacks on PKCS#7 padding
• Key Derivation: Weak passwords, predictable IVs
• Implementation Flaws: Custom crypto, weak randomness

Design Tips:

• Clearly specify the mode and parameters used
• Provide an oracle or service for interactive challenges
• Ensure attacks are feasible computationally

3. Asymmetric Cryptography (RSA)

Good for: Medium to hard challenges

Common Attacks:

• Factorization: Small primes, Fermat's method, Pollard's p-1
• Small Exponents: Low public exponent attacks, Håstad's broadcast attack
• Common Modulus: Multiple messages with same n but different e
• Wiener's Attack: Small private exponent d
• Coppersmith's Method: Known plaintext bits, related messages
• LSB Oracle: Chosen ciphertext attack revealing plaintext bits

4. Elliptic Curve Cryptography

Good for: Medium to expert challenges

Common Attacks:

• Invalid Curve: Points not on the curve, small subgroup attacks
• Nonce Reuse: ECDSA signature vulnerabilities
• MOV Attack: Weak curves reducing ECDLP to DLP
• Smart's Attack: Anomalous curves over small fields

5. Hash Functions

Good for: All difficulty levels

Common Types:

• Length Extension: Merkle-Damgård construction vulnerabilities
• Collision Finding: Birthday paradox, chosen-prefix collisions
• Hash Cracking: Rainbow tables, dictionary attacks
• HMAC Timing: Side-channel attacks on authentication

6. Post-Quantum Cryptography

Good for: Expert/insane challenges

Topics:

• Lattice-based schemes (LWE, NTRU)
• Code-based cryptography (McEliece)
• Hash-based signatures (SPHINCS+)
• Multivariate cryptography (Rainbow)

Writing Challenge Descriptions

Components of a Good Description

1. Context/Story (Optional but Engaging)

You've intercepted an encrypted message from a criminal organization. 
Intelligence suggests they're using a weak encryption scheme. Can you 
decrypt it before their next operation?

2. Technical Information (Required)

• What algorithm or protocol is being used
• Any relevant parameters (key size, modulus, etc.)
• What the player needs to find (decrypt message, forge signature, etc.)

3. Given Materials

• Source code (if applicable)
• Ciphertext or encrypted data
• Public keys or parameters
• Connection information for services

4. Flag Format (Always Include)

Flag format: flag{...} or CTF{...} or uiuc{...}

Example Descriptions by Difficulty

Beginner Example:

Medium Example:

Designing an Effective Hint System

Hint Philosophy

Hints should guide players toward the solution without giving it away entirely. Each hint should provide progressively more specific information.

Hint Tiers

Tier 1: Direction (What to Look For)

Points players to the general attack category or vulnerability.

Example: "The modulus can be factored using well-known tools."

Tier 2: Approach (How to Start)

Suggests specific tools or techniques.

Example: "Try using factordb.com or yafu to factor the modulus."

Tier 3: Implementation Detail

Provides specific steps or code snippets.

Example: "After factoring n into p and q, calculate φ(n) = (p-1)(q-1), 
then find d = e^(-1) mod φ(n) using the extended Euclidean algorithm."

Hint Timing

• Beginner: 1 hint that's fairly direct
• Easy: 1-2 hints pointing to the attack type and tools
• Medium: 2-3 hints progressively revealing the solution
• Hard: 2-3 hints that still require significant work
• Expert: 3+ hints that clarify the problem but don't solve it

Flag Format and Submission

Standard Flag Formats

• flag{...} - Generic format
• CTF{...} - Competition-specific
• uiuc{...} - University/organization-specific

Flag Content Types

1. Plaintext Messages

flag{this_is_the_decrypted_message}

Best for: Decryption challenges, encoding puzzles

2. Hexadecimal Values

flag{deadbeef1337c0de}

Best for: Binary data, hash outputs

3. Computed Values

flag{12345678901234567890}

Best for: Mathematical solutions, private keys

Flag Generation Best Practices

• Make flags unambiguous (avoid similar characters like O/0, l/1)
• Use lowercase for consistency
• Clearly specify the expected format in the challenge description
• Consider case-insensitive flag checking when appropriate
• Avoid spaces unless absolutely necessary
• For hash challenges, specify the algorithm and format (e.g., "MD5 in hex")

Dynamic Flags

For larger competitions, consider dynamic flags where each team gets a unique flag:

• Prevents flag sharing between teams
• Requires server-side infrastructure
• Must ensure all instances are equally difficult

Testing and Validation

Testing Checklist

• Solve the challenge yourself from scratch
• Have someone at the target skill level test it
• Verify the challenge is solvable in the estimated time
• Check for unintended solutions
• Test all provided files and scripts
• Verify connection details for networked challenges
• Ensure hints are accurate and helpful
• Confirm the flag format is correct

Common Issues to Check

1. Mathematical Accuracy

• All cryptographic parameters are valid
• Modular arithmetic is correct
• No typos in large numbers

2. Code Quality

• Source code runs without errors
• Dependencies are documented
• No debug output that spoils the solution

3. Difficulty Calibration

• Time to solve matches the difficulty rating
• Required knowledge is appropriate for the category
• No impossible steps or dead ends

4. Accessibility

• Challenge is solvable without expensive tools
• Computational requirements are reasonable
• Network services are stable and accessible

Writing Comprehensive Writeups

Why Write Writeups?

• Help players learn from challenges they couldn't solve
• Provide reference material for similar problems
• Document your intended solution path
• Build your reputation as a challenge author

Writeup Structure

1. Challenge Overview

• Brief description of the challenge
• Difficulty level and point value
• Category (crypto, reversing, web, etc.)

2. Initial Analysis

• What's immediately apparent from the challenge?
• What files or information are provided?
• What's the goal (decrypt, forge, crack, etc.)?

3. Identifying the Vulnerability

• What weakness or attack vector exists?
• Why is this exploitable?
• What prerequisites are needed (tools, knowledge)?

4. Step-by-Step Solution

• Detailed explanation of each step
• Code snippets with comments
• Output from intermediate steps
• Explanation of why each step works

5. Alternative Solutions

• Mention other valid approaches
• Discuss unintended solutions if they occurred
• Compare efficiency of different methods

6. Learning Points

• Key cryptographic concepts demonstrated
• How to recognize similar vulnerabilities
• Real-world implications
• Mitigation techniques

Example Writeup Excerpt

from Crypto.Util.number import inverse, long_to_bytes
import sympy

# Factor the modulus
n = 0x00c4f5f6a6e7d8e9...
factors = sympy.factorint(n)
p, q = list(factors.keys())

# Calculate private exponent
phi = (p-1) * (q-1)
d = inverse(e, phi)

# Decrypt
m = pow(c, d, n)
flag = long_to_bytes(m).decode()
print(f"Flag: {flag}")

Common Pitfalls to Avoid

1. Guessing Games

Problem: Challenge requires players to guess obscure details

Example: "The key is my favorite number between 1 and 1000000"

Solution: Provide clear constraints or hints that narrow possibilities

2. Unreasonable Computation

Problem: Solution requires days of computation on standard hardware

Example: Brute-forcing 80-bit keys without hints

Solution: Ensure challenges are solvable in hours, not days

3. Broken Infrastructure

Problem: Network services crash or have rate limiting issues

Solution: Stress test your services, implement proper error handling

4. Ambiguous Requirements

Problem: Players don't know what format to submit

Example: "Submit the answer" (as hex? decimal? ASCII?)

Solution: Always specify exact flag format

5. Unintended Solutions

Problem: Challenge can be solved without understanding the concept

Example: Source code contains hardcoded flag

Solution: Review all materials for information leakage

6. Insufficient Information

Problem: Critical information is missing or unclear

Example: Not specifying the cipher mode or padding scheme

Solution: Provide all necessary technical details

7. Real-World Dangerous Code

Problem: Challenge accidentally teaches insecure practices

Example: Showing how to implement crypto without warnings

Solution: Include warnings about not using custom crypto in production

8. Overly Specific Tools

Problem: Challenge requires rare or expensive software

Example: "Solve this using IDA Pro" (expensive tool)

Solution: Ensure solutions work with free, open-source tools

Advanced Challenge Design Topics

1. Multi-Stage Challenges

Challenges where solving one part reveals the next:

• First decrypt outer layer to get second ciphertext
• Use information from first stage to solve second
• Final flag requires combining information from all stages

Benefits: More engaging, tests multiple skills

Caution: Each stage should be achievable; avoid making early stages too hard

2. Interactive Challenges

Challenges requiring interaction with a service:

• Padding oracle servers
• Signature verification services
• Timed challenges with rate limits

Implementation Considerations:

• Rate limiting to prevent abuse
• Proper error handling
• Connection timeouts
• Resource limits (CPU, memory)
• Docker containers for isolation

3. Side-Channel Challenges

Exploiting timing, power, or other side channels:

• Timing attacks on authentication
• Cache-timing attacks
• Power analysis simulation

Design Tips:

• Simulate side channels clearly (don't require actual hardware)
• Provide data collection scripts
• Make timing differences significant

4. Protocol-Breaking Challenges

Attacking cryptographic protocols rather than primitives:

• TLS downgrade attacks
• Authentication bypass
• Key exchange vulnerabilities

5. Code Analysis Challenges

Finding vulnerabilities in implementation:

• Custom crypto implementations with subtle bugs
• Weak random number generation
• Integer overflow in crypto code
• Timing vulnerabilities

6. Mathematical Challenges

Heavy focus on number theory and algebra:

• Lattice reduction problems
• Discrete logarithm variants
• Polynomial reconstruction
• Chinese Remainder Theorem applications

7. Real-World CVE Recreation

Simplified versions of actual vulnerabilities:

• Heartbleed-style buffer over-reads
• POODLE attack simulation
• Bleichenbacher's attack on RSA

Benefits: Teaches real security history

Caution: Simplify appropriately for CTF timeframe

Resources for Challenge Authors

Cryptographic Libraries

• Python: pycryptodome, cryptography, sage
• Go: crypto/* standard library packages
• Rust: ring, rust-crypto
• C/C++: OpenSSL, libsodium

Testing and Analysis Tools

• CyberChef: Web-based encoding/decoding
• RsaCtfTool: Automated RSA attack tool
• factordb.com: Large integer factorization database
• SageMath: Mathematical software for cryptanalysis
• hashcat: Password cracking
• John the Ripper: Password cracking

Learning Resources

• Cryptopals: Comprehensive crypto challenges
• CryptoHack: Modern crypto learning platform
• CTFtime: Archive of past CTF challenges
• IACR ePrint: Academic crypto papers

Reference Materials

• NIST Cryptography Standards
• IETF RFCs
• Bruce Schneier's Blog
• Cryptography Stack Exchange

Vulnerability Databases

• CVE Database: Real-world crypto vulnerabilities
• CWE: Common weakness enumeration
• OWASP: Web application security

CTF Platforms

• CTFd: Open-source CTF platform
• rCTF: Modern, scalable platform
• HackTheBox: Continuous platform
• picoCTF: Educational platform

Quick Reference: Best Practices

Conclusion

Creating excellent cryptography CTF challenges is both an art and a science. The best challenges:

• Teach valuable concepts that transfer to real-world security
• Are fair and well-balanced for their difficulty level
• Provide clear paths to solution without being trivial
• Respect players' time with reasonable computational requirements
• Include comprehensive documentation and writeups

Remember that every challenge you create is an opportunity to educate and inspire the next generation of security professionals. Take pride in your work, test thoroughly, and always prioritize the learning experience.

Happy challenge creating!