Understanding Data Storage for Cyfin: Cloud and On-Premises Environments
When using Cyfin, it’s important to understand how we handle and store data, whether you’re running in our cloud environment or deploying an on-premises version. This article outlines the key points for both types of deployments and explains the storage of processed and enhanced data—not raw log data.
Data Processing and Storage in Cyfin
Cyfin’s primary function is to receive a syslog data stream from firewalls or other supported sources, and then process, parse, and enhance this data. During this process, Cyfin only ingests the specific fields required for analysis and reporting. Once the data is ingested, we apply compression and store only the valuable, enriched data used for generating employee-focused web usage reports that are manager and HR-friendly. Importantly, Cyfin does not store the original raw log files sent to us; instead, we store the processed and optimized data ready for reporting and analysis.
Cloud Environment Customers: Storage Options and Costs
If you are running Cyfin in our cloud environment, your data will be securely stored within the Cyfin metric cloud ecosystem. We offer two types of storage based on your operational needs:
- Cold Storage: This option is suitable for customers who do not require real-time access to their data. It offers a lower cost per gigabyte, though retrieval times may be slower. Cold storage is ideal for customers who initiate investigations based on internal requests rather than constant monitoring.
- Hot Storage: For customers who need faster access to their data, hot storage provides quicker retrieval times at a higher per-gigabyte rate. This option is best suited for customers who require immediate access to the data for monitoring or investigations.
You also have the flexibility to combine both hot and cold storage to optimize cost and performance based on your retention policy. For example, if your retention policy is one year, you could use hot storage for the first three months to ensure rapid access to more recent data for investigations, and cold storage for the remaining nine months, where quick access is less critical. This hybrid approach allows you to balance speed and cost depending on your data access needs over time.
Since each customer’s storage requirements can vary based on web resource usage and the fields we ingest, we recommend starting with a proof of concept to determine the actual storage required. To obtain an accurate storage estimate for your retention policy (such as one year), you will need to syslog your live data stream and allow Cyfin to ingest data for a sufficient period. This will give you a clearer idea of your peak and slow ingestion periods and help you properly estimate long-term storage needs.
On-Premises VM Customers: Provisioning Your Own Storage
For customers installing Cyfin on an on-premises virtual machine (VM), you will need to provision storage in your own environment. Similar to our cloud offering, you will be responsible for storing the processed and enhanced data, not the raw log files. We recommend evaluating the product by syslogging your live data stream over a period long enough to capture both peak and slow periods. This will help you estimate the storage needed to retain data for your required retention policy, typically one year.
Just like with our cloud-based offering, on-premises customers can also opt to combine hot and cold storage for a more tailored solution. For instance, you might provision hot storage for data you need immediate access to and cold storage for older data that you don’t access frequently, helping you manage costs without sacrificing performance when it matters most.
Conclusion
Understanding the storage requirements for Cyfin is essential for optimizing both cost and performance. Whether you choose our cloud-based storage options or an on-premises deployment, rest assured that only the processed and enhanced data is stored, and we can tailor storage solutions to fit your specific needs. You also have the flexibility to combine hot and cold storage to balance retrieval speed and cost over time. To accurately assess storage for your retention policy, it’s critical to evaluate Cyfin by ingesting live data over a suitable period. If you need assistance calculating storage requirements or determining the best storage solution for your setup, feel free to contact our support team.