Flat NDJSON Output Format

Newline-delimited JSON optimized for streaming and log analysis

Overview

The Flat NDJSON format outputs one JSON record per line, with each line representing a single cipher suite, certificate, crypto library, or other cryptographic asset. All nested structures are flattened using dot notation.

Best For

  • • ELK Stack ingestion
  • • Streaming log analysis
  • • Time-series databases
  • • Log aggregation systems

Usage

`.\certscanner-windows-amd64.exe -host example.com `
  -outputformat flatndjson `
  -output stream.ndjson
./certscanner-linux-x64 -host example.com \
  -outputformat flatndjson \
  -output stream.ndjson
# Intel Macs
./certscanner-darwin-amd64 -host example.com \
  -outputformat flatndjson \
  -output stream.ndjson

# Apple Silicon Macs
./certscanner-darwin-arm64 -host example.com \
  -outputformat flatndjson \
  -output stream.ndjson

Complete Flat NDJSON Schema

Schema Overview

Each line in the NDJSON output represents a single cryptographic asset event. All nested JSON structures are flattened using dot notation (e.g., certificate.subject.common_name). Different event types share common base fields but include type-specific fields.

Base Event Fields (All Events)

Field Type Required Description Example
@timestampStringYesEvent timestamp (ISO 8601)"2025-09-02T13:45:30.123Z"
event.actionStringYesEvent type identifier"cipher_suite_discovered"
event.categoryStringYesECS event category"host", "network"
event.typeStringYesECS event type"info", "connection"
observer.hostnameStringYesScanning system hostname"scanner-host.company.com"
observer.ipArray<String>NoScanner IP addresses["192.168.1.100"]
observer.os.nameStringYesScanner OS name"Windows", "Linux", "macOS"
observer.os.versionStringNoScanner OS version"10.0.19045"
observer.fips_mode_enabledBooleanYesFIPS 140-2 mode statustrue, false
observer.bigfix_client_installedBooleanNoIndicates if BigFix client is installedtrue, false
observer.bigfix_client_idStringNoBigFix client ID for asset correlation"12345678"
observer.software.nameStringYesTool name"TYCHON Quantum Readiness"
observer.software.versionStringYesTool version"1.0.42"
scan.typeStringYesScan mode"local", "remote"
scan.targetStringYesOriginal target specification"example.com:443"
tagsArray<String>NoCustom scan tags["prod", "compliance"]

Quantum Readiness Fields (Local Mode Only) NEW

These fields are added to all events when quantum readiness assessment is enabled in local mode (default). Can be disabled with -disable-quantum-readiness flag.

Field Type Description Example
quantum.assessment_idStringUnique assessment identifier"qr_20250915_101539_abc123"
quantum.timestampStringAssessment timestamp (ISO 8601)"2025-09-15T10:15:39.123456-07:00"
quantum.assessment_typeStringType of assessment performed"comprehensive"
quantum.system_typeStringClassification of system type"workstation"
quantum.system_roleStringPrimary role of the system"workstation"
quantum.criticality_levelStringSystem criticality classification"standard"
quantum.fips_mode_enabledBooleanFIPS 140-2 mode status at assessment timetrue, false
quantum.overall_scoreIntegerTotal quantum readiness score (0-100)64
quantum.readiness_statusStringOverall readiness classification"Update Required"
quantum.status_colorStringColor code for status visualization"orange"
quantum.ready_timelineStringEstimated timeline to quantum readiness"2-6 months"
quantum.hardware_score.total_scoreIntegerHardware assessment score (0-40)32
quantum.hardware_score.cpu_scoreIntegerCPU assessment score (0-20)18
quantum.hardware_score.memory_scoreIntegerMemory assessment score (0-15)14
quantum.hardware_score.security_hardware_scoreIntegerSecurity hardware score (0-5)0
quantum.operating_system_score.total_scoreIntegerOS assessment score (0-30)15
quantum.operating_system_score.version_scoreIntegerOS version score (0-20)12
quantum.operating_system_score.crypto_framework_scoreIntegerCrypto framework score (0-10)3
quantum.crypto_library_score.total_scoreIntegerCrypto library score (0-25)12
quantum.crypto_library_score.openssl_scoreIntegerOpenSSL assessment score (0-15)8
quantum.crypto_library_score.system_crypto_scoreIntegerSystem crypto score (0-10)4
quantum.network_score.total_scoreIntegerNetwork assessment score (0-5)5
quantum.network_score.bandwidth_scoreIntegerBandwidth score (0-3)3
quantum.network_score.protocol_capability_scoreIntegerProtocol capability score (0-2)2
quantum.recommendationsArray<String>Actionable recommendations["Upgrade to macOS 15.0+"]
quantum.detailed_reportStringComprehensive assessment summary"System shows moderate quantum readiness..."

Network Target Fields (Remote Scans)

Field Type Description Example
destination.addressStringTarget hostname or IP"example.com"
destination.ipStringResolved IP address"203.0.113.1"
destination.domainStringDomain name"example.com"
destination.portIntegerTarget port number443
network.protocolStringNetwork protocol"tcp"
network.transportStringTransport protocol"tcp"
service.nameStringDetected service"https", "ssh"
service.versionStringService version (if detected)"TLSv1.3"

Cipher Suite Event Fields

Event Action: cipher_suite_discovered

Field Type Description Example
tls.versionStringTLS protocol version"TLSv1.3"
tls.cipherStringIANA cipher suite name"TLS_AES_256_GCM_SHA384"
tls.cipher_hexStringHex identifier"0x13,0x02"
tls.key_lengthIntegerKey length in bits256
tls.negotiated_groupStringKey exchange group"X25519"
tls.is_preferredBooleanServer's preferred choicetrue
tls.openssl_nameStringOpenSSL cipher name"ECDHE-RSA-AES256-GCM-SHA384"
security.levelStringSecurity assessment"high", "medium", "low"
security.recommendationStringSecurity recommendation"recommended", "legacy"
security.pqc_readyBooleanPost-quantum readyfalse
security.vulnerabilitiesArray<String>Known vulnerabilities["BEAST", "CRIME"]
security.nist_categoryStringNIST classification"Recommended"

Certificate Event Fields

Event Actions: network_certificate_discovered, filesystem_certificate_discovered

Field Type Description Example
x509.subject.common_nameStringCertificate subject CN"example.com"
x509.subject.organizationStringSubject organization"Example Corp"
x509.subject.organizational_unitStringSubject OU"IT Department"
x509.subject.countryStringSubject country"US"
x509.subject.state_or_provinceStringSubject state/province"California"
x509.subject.localityStringSubject locality"San Francisco"
x509.subject.email_addressStringSubject email"admin@example.com"
x509.subject.rawStringComplete subject DN"CN=example.com,O=Example Corp,C=US"
x509.issuer.common_nameStringIssuer CN"DigiCert TLS RSA SHA256 2020 CA1"
x509.issuer.organizationStringIssuer organization"DigiCert Inc"
x509.issuer.countryStringIssuer country"US"
x509.issuer.rawStringComplete issuer DN"CN=DigiCert TLS RSA SHA256 2020 CA1,O=DigiCert Inc,C=US"
x509.serial_numberStringCertificate serial number"123456789012345678901234567890"
x509.signature_algorithmStringSignature algorithm"SHA256-RSA"
x509.public_key.algorithmStringPublic key algorithm"RSA", "ECDSA"
x509.public_key.sizeIntegerPublic key size in bits2048
x509.public_key.curveStringCurve name (ECDSA)"secp384r1"
x509.not_beforeStringValid from date (ISO 8601)"2024-03-01T00:00:00Z"
x509.not_afterStringValid until date (ISO 8601)"2025-03-01T23:59:59Z"
x509.is_expiredBooleanWhether cert is expiredfalse
x509.days_until_expiryIntegerDays until expiration182
x509.is_self_signedBooleanSelf-signed certificatefalse
x509.is_caBooleanCA certificatefalse
x509.fingerprint.sha256StringSHA-256 fingerprint"ab:cd:ef:12:34:..."
x509.fingerprint.sha1StringSHA-1 fingerprint"12:34:56:78:..."
x509.sanArray<String>Subject alt names["*.example.com"]
x509.key_usageArray<String>Key usage extensions["digitalSignature"]
x509.extended_key_usageArray<String>Extended key usage["serverAuth"]

Filesystem Event Fields

Event Action: filesystem_certificate_discovered

Field Type Description Example
file.pathStringFull path to certificate file"/etc/ssl/certs/ca.pem"
file.nameStringFile name only"ca.pem"
file.extensionStringFile extension".pem"
file.sizeIntegerFile size in bytes2048
file.createdStringFile creation time"2024-01-15T10:30:00Z"
file.modifiedStringLast modification time"2024-01-15T10:30:00Z"
file.ownerStringFile owner"root", "DOMAIN\\user"
file.permissionsStringFile permissions"644", "rw-r--r--"
file.hash.sha256StringSHA-256 hash of file"a1b2c3d4..."
certificate.formatStringCertificate format"PEM", "DER", "PKCS12"
certificate.typeStringCertificate type"x509", "pkcs12"

Memory Library Event Fields

Event Action: crypto_library_discovered

Field Type Description Example
process.pidIntegerProcess ID1234
process.nameStringProcess executable name"nginx.exe"
process.executableStringFull executable path"C:\\nginx\\nginx.exe"
process.command_lineStringComplete command line"nginx.exe -c nginx.conf"
process.usernameStringProcess owner"SYSTEM"
library.nameStringCrypto library name"OpenSSL"
library.versionStringLibrary version"3.0.8"
library.pathStringLibrary file path"C:\\openssl\\libssl.dll"
library.crypto_typeStringCrypto implementation type"openssl", "bcrypt", "java_crypto"
library.product_nameStringProduct name from metadata"OpenSSL Toolkit"
library.company_nameStringCompany from metadata"The OpenSSL Project"
library.file_descriptionStringFile description"OpenSSL Shared Library"
library.hash.sha256StringSHA-256 hash of library"e1f2a3b4..."

Keystore Event Fields NEW

Event Actions: keystore_discovered, keystore_certificate_discovered

Field Type Description Example
keystore.pathStringFull path to keystore file"/home/user/keystore.p12"
keystore.typeStringKeystore format type"PKCS12", "JKS", "Windows", "macOS"
keystore.accessibleBooleanWhether keystore is accessibletrue
keystore.requires_authBooleanWhether authentication requiredfalse
keystore.cert_countIntegerNumber of certificates found15
keystore.ownerStringFile owner (if available)"domain\\username"
keystore.permissionsStringFile permissions"rw-r--r--"
keystore.sizeIntegerFile size in bytes2048576
keystore.last_modifiedStringLast modification timestamp"2024-12-01T10:30:00Z"
keystore.error_messageStringError details if access failed"Password required"
keystore_certificate.aliasStringCertificate alias in keystore"my-server-cert"
keystore_certificate.has_private_keyBooleanWhether private key availabletrue
keystore_certificate.chain_lengthIntegerCertificate chain length3
keystore_certificate.chain_completeBooleanWhether chain is completetrue
keystore_certificate.vulnerableBooleanWhether has vulnerabilitiesfalse
keystore_certificate.risk_levelStringRisk assessment level"low", "medium", "high"
keystore_certificate.pqc_vulnerableBooleanQuantum vulnerability statustrue
keystore_certificate.pqc_reasonStringReason for PQC vulnerability"RSA algorithm vulnerable"

SSH Host Key Event Fields

Event Action: ssh_host_key_discovered

Field Type Description Example
ssh.host_key.algorithmStringSSH key algorithm"ssh-rsa", "ecdsa-sha2-nistp256"
ssh.host_key.sizeIntegerKey size in bits2048
ssh.host_key.curveStringElliptic curve (ECDSA)"nistp256"
ssh.host_key.fingerprint.md5StringMD5 fingerprint (legacy)"12:34:56:78:..."
ssh.host_key.fingerprint.sha256StringSHA-256 fingerprint"SHA256:abcd..."
ssh.host_key.public_keyStringBase64 public key data"AAAAB3NzaC1yc2E..."
ssh.host_key.is_weakBooleanCryptographically weak keyfalse
ssh.bannerStringSSH server banner"SSH-2.0-OpenSSH_8.9"
ssh.server_versionStringSSH server software"OpenSSH_8.9"

Outlook Archive Event Fields

Event Action: outlook_archive_discovered

Field Type Description Example
file.pathStringPath to PST/OST file"C:\\Users\\user\\archive.pst"
file.sizeIntegerArchive file size1048576000
outlook.is_encryptedBooleanArchive encryption statustrue
outlook.encryption_typeStringEncryption method"Compressible", "High"
outlook.versionStringOutlook version"2019", "365"
user.nameStringArchive owner username"john.doe"

VPN Client Event Fields NEW

Event Action: vpn_client_discovered

Field Type Description Example
vpn.client_nameStringVPN client application name"Palo Alto GlobalProtect"
vpn.vendorStringSoftware vendor"Palo Alto Networks"
vpn.versionStringClient version"6.3.2-525"
vpn.install_pathStringInstallation directory"/Applications/GlobalProtect.app"
vpn.config_pathStringConfiguration file location"~/Library/Application Support/..."
vpn.executable_pathStringMain executable path"/Applications/.../GlobalProtect"
vpn.service_nameStringSystem service identifier"com.paloaltonetworks.globalprotect"
vpn.statusStringCurrent operational status"active", "inactive", "unknown"
vpn.detection_methodStringHow client was discovered"filesystem", "registry", "process"
vpn.detection_confidenceStringDetection accuracy level"high", "medium", "low"
vpn.pqc_readyBooleanPost-quantum cryptography supporttrue
vpn.quantum_resistanceStringLevel of quantum resistance"high", "medium", "low", "none"
vpn.pqc_migration_statusStringPQC migration readiness"ready", "partial", "not_ready"
vpn.supported_pqc_algorithmsArray<String>Supported PQC algorithms["ML-KEM-512", "ML-DSA-44"]
process.pidIntegerProcess ID (if running)4473

IPSec Tunnel Event Fields NEW

Event Action: ipsec_tunnel_discovered

Field Type Description Example
ipsec.tunnel_nameStringIPSec tunnel identifier"strongSwan Site-to-Site"
ipsec.implementationStringIPSec implementation type"strongswan", "libreswan", "macOS"
ipsec.config_pathStringConfiguration file location"/etc/ipsec.conf"
ipsec.statusStringCurrent tunnel status"active", "inactive", "unknown"
ipsec.detection_methodStringHow tunnel was discovered"config_file", "process", "kernel"
ipsec.detection_confidenceStringDetection accuracy level"high", "medium", "low"
ipsec.local_subnetStringLocal network subnet"192.168.1.0/24"
ipsec.remote_subnetStringRemote network subnet"10.0.0.0/24"
ipsec.gatewayStringRemote gateway IP address"203.0.113.1"
ipsec.encryption_algorithmsArray<String>Configured encryption algorithms["aes256", "aes128"]
ipsec.integrity_algorithmsArray<String>Configured hash algorithms["sha256", "sha1"]
ipsec.key_exchange_groupsArray<String>Configured DH groups["modp2048", "ecp256"]
ipsec.pqc_readyBooleanPost-quantum cryptography supportfalse
ipsec.quantum_resistanceStringLevel of quantum resistance"high", "medium", "low", "none"
ipsec.pqc_migration_statusStringPQC migration readiness"ready", "partial", "not_ready"

Security Intelligence Fields

Applied to cipher suites and cryptographic assets

Field Type Description Possible Values
intel.security_levelStringOverall security assessment"high", "medium", "low", "insecure"
intel.recommendationStringSecurity recommendation"recommended", "acceptable", "legacy", "avoid"
intel.pqc_readyBooleanPost-quantum readyfalse
intel.pqc_vulnerableBooleanQuantum vulnerabletrue
intel.vulnerabilitiesArray<String>Known vulnerabilities["BEAST", "CRIME", "POODLE"]
intel.nist_categoryStringNIST security category"Recommended", "Legacy-Use", "Deprecated"
intel.friendly_nameStringHuman-readable name"AES-256 with GCM and SHA-384"
intel.descriptionStringDetailed description"Advanced Encryption Standard with..."
intel.risk_scoreIntegerNumeric risk score (0-100)25

Event Action Types

Network Discovery Events

  • cipher_suite_discovered
  • network_certificate_discovered
  • ssh_host_key_discovered
  • tls_handshake_completed
  • protocol_detected

Local Discovery Events

  • filesystem_certificate_discovered
  • crypto_library_discovered
  • outlook_archive_discovered
  • private_key_discovered
  • java_keystore_discovered
  • keystore_discovered NEW
  • keystore_certificate_discovered NEW
  • vpn_client_discovered NEW
  • ipsec_tunnel_discovered NEW

Common Field Patterns

Dot Notation Examples

x509.subject.common_name → Certificate CN
process.executable → Process path
intel.security_level → Risk assessment
file.hash.sha256 → File hash
tls.cipher → Cipher suite name

Array Field Handling

Multiple Values: Arrays become comma-separated
x509.san: "*.example.com,example.com"
tags: "production,compliance,audit"
vulnerabilities: "BEAST,CRIME"

Sample Records

Cipher Suite Record

{"observer.hostname":"scanner-host","observer.software_version":"1.0.42","scan.type":"remote","scan.timestamp":"2025-09-02T09:00:17-04:00","target_host.address":"example.com","target_host.ip":"93.184.216.34","port.number":443,"port.status_overall":"open","port.protocol_detected":"TLS","cipher.protocol":"TLSv1.3","cipher.cipher_suite":"TLS_AES_256_GCM_SHA384","cipher.key_length_bits":256,"cipher.negotiated_group":"X25519","cipher.is_preferred":true,"cipher.intel.security_level":"high","cipher.intel.recommendation":"recommended","x509.subject.distinguished_name":"CN=example.com,O=Example Corp,C=US","x509.serial_number":"123456789012345678901234567890","hash.sha256_certificate":"ab:cd:ef:12:34:56:78:90:..."}

Filesystem Certificate Record

{"observer.hostname":"scanner-host","scan.type":"local","event.action":"filesystem_certificate_discovered","file.path":"/etc/ssl/certs/ca-cert.pem","certificate.subject.common_name":"Internal Root CA","certificate.issuer.common_name":"Internal Root CA","certificate.serial_number":"123456789","x509.is_valid":true,"hash.sha256_certificate":"12:34:56:78:90:ab:cd:ef:..."}

Crypto Library Record

{"observer.hostname":"scanner-host","scan.type":"local","event.action":"crypto_library_in_memory","process.pid":1234,"process.name":"nginx","process.executable":"/usr/sbin/nginx","cryptolibrary.name":"libssl.so.3","cryptolibrary.path":"/usr/lib/x86_64-linux-gnu/libssl.so.3","cryptolibrary.crypto_type":"TLS Library","cryptolibrary.detected_apis":"SSL_connect,SSL_accept,TLS_method"}

Keystore Certificate Record NEW

{"@timestamp":"2025-09-17T14:30:00.000Z","observer.hostname":"scanner-host","scan.type":"local","event.action":"keystore_certificate_discovered","event.category":"file","event.dataset":"keystore_certificate","keystore.path":"/Users/admin/Documents/certificates/server.p12","keystore.type":"PKCS12","keystore.accessible":true,"keystore.cert_count":3,"keystore_certificate.alias":"server-cert","keystore_certificate.has_private_key":true,"keystore_certificate.chain_length":2,"x509.subject.distinguished_name":"CN=api.example.com,O=Example Corp,C=US","x509.issuer.distinguished_name":"CN=Example Internal CA,O=Example Corp,C=US","x509.serial_number":"0x1a2b3c4d5e6f7890","x509.public_key.algorithm":"RSA","x509.public_key.size":2048,"x509.signature_algorithm":"SHA256-RSA","x509.not_before":"2024-01-01T00:00:00Z","x509.not_after":"2025-12-31T23:59:59Z","keystore_certificate.vulnerable":false,"keystore_certificate.risk_level":"medium","keystore_certificate.pqc_vulnerable":true,"keystore_certificate.pqc_reason":"RSA algorithm vulnerable to quantum cryptanalysis"}

Key Features

Data Completeness

  • Complete Coverage: All JSON data included
  • No Field Filtering: Every certificate field exported
  • Certificate Chains: Full chain data flattened
  • Process Information: Complete process metadata

Flattening Approach

  • Dot Notation: Nested objects flattened with dots
  • Array Handling: String arrays joined with commas
  • Time Formatting: ISO 8601 RFC3339Nano format
  • Intel Maps: Recursively flattened key-value data

Event Types

Network Cipher Events

One record per cipher suite negotiated on network ports

cipher.* fields + certificate.* fields + target_host.* fields

Filesystem Certificate Events

One record per certificate file discovered

event.action="filesystem_certificate_discovered" + certificate.* fields

Memory Crypto Library Events

One record per crypto library found in process memory

event.action="crypto_library_in_memory" + cryptolibrary.* fields

Java Crypto Library Events

One record per Java crypto library discovered

event.action="java_crypto_library_in_memory" + library.* fields

Keystore Certificate Events NEW

One record per certificate found in keystores (PKCS12, JKS, System Stores)

event.action="keystore_certificate_discovered" + keystore.* + x509.* fields

Integration Examples

Elasticsearch Ingestion

# Direct streaming to Elasticsearch
`.\certscanner-windows-amd64.exe -host internal-network.txt -cipherscan `
  -outputformat flatndjson `
  -posttoelastic -elasticnode "https://elastic.company.com:9200" `
  -elasticindex "crypto-scans"

# File-based ingestion
`.\certscanner-windows-amd64.exe -mode local -scanfilesystem `
  -outputformat flatndjson -output certs.ndjson
curl -X POST "elastic.company.com:9200/certs/_bulk" `
  -H "Content-Type: application/x-ndjson" `
  --data-binary "@certs.ndjson"
# Direct streaming to Elasticsearch
./certscanner-linux-x64 -host internal-network.txt -cipherscan \
  -outputformat flatndjson \
  -posttoelastic -elasticnode "https://elastic.company.com:9200" \
  -elasticindex "crypto-scans"

# File-based ingestion
./certscanner-linux-x64 -mode local -scanfilesystem \
  -outputformat flatndjson -output certs.ndjson
curl -X POST "elastic.company.com:9200/certs/_bulk" \
  -H "Content-Type: application/x-ndjson" \
  --data-binary "@certs.ndjson"
# Direct streaming to Elasticsearch - Intel Macs
./certscanner-darwin-amd64 -host internal-network.txt -cipherscan \
  -outputformat flatndjson \
  -posttoelastic -elasticnode "https://elastic.company.com:9200" \
  -elasticindex "crypto-scans"

# File-based ingestion - Intel Macs
./certscanner-darwin-amd64 -mode local -scanfilesystem \
  -outputformat flatndjson -output certs.ndjson
  
# For Apple Silicon Macs, use:
# ./certscanner-darwin-arm64 [same arguments]

curl -X POST "elastic.company.com:9200/certs/_bulk" \
  -H "Content-Type: application/x-ndjson" \
  --data-binary "@certs.ndjson"

Log Analysis Pipeline

# Stream processing with jq
`.\certscanner-windows-amd64.exe -host servers.txt -outputformat flatndjson | `
  jq 'select(.cipher.intel.security_level == "low")' | `
  jq '.target_host.address + ":" + (.port.number | tostring) + " - " + .cipher.cipher_suite'

# Filter for expiring certificates
`.\certscanner-windows-amd64.exe -mode local -scanfilesystem -outputformat flatndjson | `
  jq 'select(.event.action == "filesystem_certificate_discovered") | select(.x509.is_valid == false)'
# Stream processing with jq
./certscanner-linux-x64 -host servers.txt -outputformat flatndjson | \
  jq 'select(.cipher.intel.security_level == "low")' | \
  jq '.target_host.address + ":" + (.port.number | tostring) + " - " + .cipher.cipher_suite'

# Filter for expiring certificates
./certscanner-linux-x64 -mode local -scanfilesystem -outputformat flatndjson | \
  jq 'select(.event.action == "filesystem_certificate_discovered") | select(.x509.is_valid == false)'
# Stream processing with jq - Intel Macs
./certscanner-darwin-amd64 -host servers.txt -outputformat flatndjson | \
  jq 'select(.cipher.intel.security_level == "low")' | \
  jq '.target_host.address + ":" + (.port.number | tostring) + " - " + .cipher.cipher_suite'

# Filter for expiring certificates - Intel Macs (no memory scanning)
./certscanner-darwin-amd64 -mode local -scanfilesystem -outputformat flatndjson | \
  jq 'select(.event.action == "filesystem_certificate_discovered") | select(.x509.is_valid == false)'
  
# For Apple Silicon Macs, replace -darwin-amd64 with -darwin-arm64