mirror of
https://github.com/tiennm99/java-design-patterns.git
synced 2026-08-16 18:25:33 +00:00
refactor: rename microservices patterns
This commit is contained in:
@@ -0,0 +1,161 @@
|
||||
---
|
||||
title: Log Aggregation
|
||||
category: Integration
|
||||
language: en
|
||||
tag:
|
||||
- Data processing
|
||||
- Decoupling
|
||||
- Enterprise patterns
|
||||
- Fault tolerance
|
||||
- Messaging
|
||||
- Microservices
|
||||
- Performance
|
||||
- Scalability
|
||||
---
|
||||
|
||||
## Also known as
|
||||
|
||||
* Centralized Logging
|
||||
* Log Management
|
||||
|
||||
## Intent
|
||||
|
||||
Log Aggregation is a pattern that centralizes the collection, storage, and analysis of logs from multiple sources to facilitate monitoring, debugging, and operational intelligence.
|
||||
|
||||
## Explanation
|
||||
|
||||
Real-world example
|
||||
|
||||
> Consider an e-commerce platform that operates using a microservices architecture. Each service, such as user authentication, product catalog, order processing, and payment, generates its own logs. To effectively monitor and analyze the entire platform's activity, a log aggregation system is implemented. This system collects logs from each microservice and centralizes them into a single location using tools like the ELK Stack (Elasticsearch, Logstash, Kibana). This allows the platform administrators to have a unified view of all logs, enabling real-time monitoring, quick troubleshooting, and comprehensive analysis of user behavior and system performance.
|
||||
|
||||
In plain words
|
||||
|
||||
> The Log Aggregation design pattern centralizes the collection and analysis of log data from multiple applications or services to simplify monitoring and troubleshooting.
|
||||
|
||||
Wikipedia says
|
||||
|
||||
> You have applied the Microservice architecture pattern. The application consists of multiple services and service instances that are running on multiple machines. Requests often span multiple service instances. Each service instance generates writes information about what it is doing to a log file in a standardized format. The log file contains errors, warnings, information and debug messages.
|
||||
|
||||
**Programmatic example**
|
||||
|
||||
Log Aggregation is a pattern that centralizes the collection, storage, and analysis of logs from multiple sources to facilitate monitoring, debugging, and operational intelligence. It is particularly useful in distributed systems where logs from various components need to be centralized for better management and analysis.
|
||||
|
||||
In this example, we will demonstrate the Log Aggregation pattern using a simple Java application. The application consists of multiple services that generate logs. These logs are collected by a log aggregator and stored in a central log store.
|
||||
|
||||
The `CentralLogStore` is responsible for storing the logs collected from various services. In this example, we are using an in-memory store for simplicity.
|
||||
|
||||
```java
|
||||
public class CentralLogStore {
|
||||
|
||||
private final List<LogEntry> logs = new ArrayList<>();
|
||||
|
||||
public void storeLog(LogEntry logEntry) {
|
||||
logs.add(logEntry);
|
||||
}
|
||||
|
||||
public void displayLogs() {
|
||||
logs.forEach(System.out::println);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The `LogAggregator` collects logs from various services and stores them in the `CentralLogStore`. It filters logs based on their log level.
|
||||
|
||||
```java
|
||||
public class LogAggregator {
|
||||
|
||||
private final CentralLogStore centralLogStore;
|
||||
private final LogLevel minimumLogLevel;
|
||||
|
||||
public LogAggregator(CentralLogStore centralLogStore, LogLevel minimumLogLevel) {
|
||||
this.centralLogStore = centralLogStore;
|
||||
this.minimumLogLevel = minimumLogLevel;
|
||||
}
|
||||
|
||||
public void collectLog(LogEntry logEntry) {
|
||||
if (logEntry.getLogLevel().compareTo(minimumLogLevel) >= 0) {
|
||||
centralLogStore.storeLog(logEntry);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The `LogProducer` represents a service that generates logs. It sends the logs to the `LogAggregator`.
|
||||
|
||||
```java
|
||||
public class LogProducer {
|
||||
|
||||
private final String serviceName;
|
||||
private final LogAggregator logAggregator;
|
||||
|
||||
public LogProducer(String serviceName, LogAggregator logAggregator) {
|
||||
this.serviceName = serviceName;
|
||||
this.logAggregator = logAggregator;
|
||||
}
|
||||
|
||||
public void generateLog(LogLevel logLevel, String message) {
|
||||
LogEntry logEntry = new LogEntry(serviceName, logLevel, message, LocalDateTime.now());
|
||||
logAggregator.collectLog(logEntry);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The `main` application creates services, generates logs, aggregates, and finally displays the logs.
|
||||
|
||||
```java
|
||||
public class App {
|
||||
|
||||
public static void main(String[] args) throws InterruptedException {
|
||||
final CentralLogStore centralLogStore = new CentralLogStore();
|
||||
final LogAggregator aggregator = new LogAggregator(centralLogStore, LogLevel.INFO);
|
||||
|
||||
final LogProducer serviceA = new LogProducer("ServiceA", aggregator);
|
||||
final LogProducer serviceB = new LogProducer("ServiceB", aggregator);
|
||||
|
||||
serviceA.generateLog(LogLevel.INFO, "This is an INFO log from ServiceA");
|
||||
serviceB.generateLog(LogLevel.ERROR, "This is an ERROR log from ServiceB");
|
||||
serviceA.generateLog(LogLevel.DEBUG, "This is a DEBUG log from ServiceA");
|
||||
|
||||
centralLogStore.displayLogs();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
In this example, the `LogProducer` services generate logs of different levels. The `LogAggregator` collects these logs and stores them in the `CentralLogStore` if they meet the minimum log level requirement. Finally, the logs are displayed by the `CentralLogStore`.
|
||||
|
||||
## Applicability
|
||||
|
||||
* Useful in distributed systems where logs from various components need to be centralized for better management and analysis.
|
||||
* Applicable in environments where compliance and auditing require consolidated log data.
|
||||
* Beneficial in systems that require high availability and resilience, ensuring that log data is preserved and accessible despite individual component failures.
|
||||
|
||||
## Known Uses
|
||||
|
||||
* Java applications using frameworks like Log4j2 or SLF4J paired with centralized log management tools like Elasticsearch, Logstash, and Kibana (ELK stack) or Splunk.
|
||||
* Microservices architectures where each service outputs logs that are aggregated into a single system to provide a unified view of the system’s health and behavior.
|
||||
|
||||
## Consequences
|
||||
|
||||
Benefits:
|
||||
|
||||
* Improves debuggability and traceability of issues across multiple services or components.
|
||||
* Enhances monitoring capabilities by providing a centralized platform for log analysis.
|
||||
* Facilitates compliance with regulatory requirements for log retention and auditability.
|
||||
|
||||
Trade-offs:
|
||||
|
||||
* Introduces a potential single point of failure if the log aggregation system is not adequately resilient.
|
||||
* Can lead to high data volumes requiring significant storage and processing resources.
|
||||
|
||||
## Related Patterns
|
||||
|
||||
* Messaging Patterns: Log Aggregation often utilizes messaging systems to transport log data, facilitating decoupling and asynchronous data processing.
|
||||
* Microservices: Often employed in microservice architectures to handle logs from various services efficiently.
|
||||
* Publish/Subscribe: Utilizes a pub/sub model for log data collection where components publish logs and the aggregation system subscribes to them.
|
||||
|
||||
## Credits
|
||||
|
||||
* [Cloud Native Java: Designing Resilient Systems with Spring Boot, Spring Cloud, and Cloud Foundry](https://amzn.to/44vDTat)
|
||||
* [Logging in Action: With Fluentd, Kubernetes and more](https://amzn.to/3JQLzdT)
|
||||
* [Release It! Design and Deploy Production-Ready Software](https://amzn.to/3Uul4kF)
|
||||
* [Pattern: Log aggregation (microservices.io)](https://microservices.io/patterns/observability/application-logging.html)
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 68 KiB |
@@ -0,0 +1,51 @@
|
||||
@startuml
|
||||
|
||||
package com.iluwatar.logaggregation {
|
||||
|
||||
class App {
|
||||
+ main(args: String[]) {static}
|
||||
}
|
||||
|
||||
class CentralLogStore {
|
||||
- logs: ConcurrentLinkedQueue<LogEntry>
|
||||
+ storeLog(logEntry: LogEntry)
|
||||
+ displayLogs()
|
||||
}
|
||||
|
||||
class LogAggregator {
|
||||
- BUFFER_THRESHOLD: int {static}
|
||||
- centralLogStore: CentralLogStore
|
||||
- buffer: ConcurrentLinkedQueue<LogEntry>
|
||||
- minLogLevel: LogLevel
|
||||
- executorService: ExecutorService
|
||||
- logCount: AtomicInteger
|
||||
+ collectLog(logEntry: LogEntry)
|
||||
+ stop()
|
||||
}
|
||||
|
||||
class LogEntry {
|
||||
- serviceName: String
|
||||
- level: LogLevel
|
||||
- message: String
|
||||
- timestamp: LocalDateTime
|
||||
}
|
||||
|
||||
enum LogLevel {
|
||||
DEBUG
|
||||
INFO
|
||||
ERROR
|
||||
}
|
||||
|
||||
class LogProducer {
|
||||
- serviceName: String
|
||||
- aggregator: LogAggregator
|
||||
+ generateLog(level: LogLevel, message: String)
|
||||
}
|
||||
}
|
||||
|
||||
LogProducer --> "-aggregator" LogAggregator
|
||||
LogAggregator --> "-centralLogStore" CentralLogStore
|
||||
LogAggregator --> "-buffer" LogEntry
|
||||
CentralLogStore --> "-logs" LogEntry
|
||||
|
||||
@enduml
|
||||
@@ -0,0 +1,68 @@
|
||||
@startuml
|
||||
package com.iluwatar.logaggregation {
|
||||
class App {
|
||||
+ App()
|
||||
+ main(args : String[]) {static}
|
||||
}
|
||||
class CentralLogStore {
|
||||
- LOGGER : Logger {static}
|
||||
- logs : ConcurrentLinkedQueue<LogEntry>
|
||||
+ CentralLogStore()
|
||||
+ displayLogs()
|
||||
+ storeLog(logEntry : LogEntry)
|
||||
}
|
||||
class LogAggregator {
|
||||
- BUFFER_THRESHOLD : int {static}
|
||||
- LOGGER : Logger {static}
|
||||
- buffer : ConcurrentLinkedQueue<LogEntry>
|
||||
- centralLogStore : CentralLogStore
|
||||
- executorService : ExecutorService
|
||||
- logCount : AtomicInteger
|
||||
- minLogLevel : LogLevel
|
||||
+ LogAggregator(centralLogStore : CentralLogStore, minLogLevel : LogLevel)
|
||||
+ collectLog(logEntry : LogEntry)
|
||||
- flushBuffer()
|
||||
- startBufferFlusher()
|
||||
+ stop()
|
||||
}
|
||||
class LogEntry {
|
||||
- level : LogLevel
|
||||
- message : String
|
||||
- serviceName : String
|
||||
- timestamp : LocalDateTime
|
||||
+ LogEntry(serviceName : String, level : LogLevel, message : String, timestamp : LocalDateTime)
|
||||
# canEqual(other : Object) : boolean
|
||||
+ equals(o : Object) : boolean
|
||||
+ getLevel() : LogLevel
|
||||
+ getMessage() : String
|
||||
+ getServiceName() : String
|
||||
+ getTimestamp() : LocalDateTime
|
||||
+ hashCode() : int
|
||||
+ setLevel(level : LogLevel)
|
||||
+ setMessage(message : String)
|
||||
+ setServiceName(serviceName : String)
|
||||
+ setTimestamp(timestamp : LocalDateTime)
|
||||
+ toString() : String
|
||||
}
|
||||
enum LogLevel {
|
||||
+ DEBUG {static}
|
||||
+ ERROR {static}
|
||||
+ INFO {static}
|
||||
+ valueOf(name : String) : LogLevel {static}
|
||||
+ values() : LogLevel[] {static}
|
||||
}
|
||||
class LogProducer {
|
||||
- LOGGER : Logger {static}
|
||||
- aggregator : LogAggregator
|
||||
- serviceName : String
|
||||
+ LogProducer(serviceName : String, aggregator : LogAggregator)
|
||||
+ generateLog(level : LogLevel, message : String)
|
||||
}
|
||||
}
|
||||
LogAggregator --> "-centralLogStore" CentralLogStore
|
||||
LogEntry --> "-level" LogLevel
|
||||
CentralLogStore --> "-logs" LogEntry
|
||||
LogAggregator --> "-buffer" LogEntry
|
||||
LogAggregator --> "-minLogLevel" LogLevel
|
||||
LogProducer --> "-aggregator" LogAggregator
|
||||
@enduml
|
||||
@@ -0,0 +1,58 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!--
|
||||
|
||||
This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
|
||||
|
||||
The MIT License
|
||||
Copyright © 2014-2022 Ilkka Seppälä
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
|
||||
-->
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
<parent>
|
||||
<groupId>com.iluwatar</groupId>
|
||||
<artifactId>java-design-patterns</artifactId>
|
||||
<version>1.26.0-SNAPSHOT</version>
|
||||
</parent>
|
||||
|
||||
<artifactId>microservices-log-aggregation</artifactId>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.junit.jupiter</groupId>
|
||||
<artifactId>junit-jupiter-engine</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.mockito</groupId>
|
||||
<artifactId>mockito-junit-jupiter</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
<properties>
|
||||
<maven.compiler.source>17</maven.compiler.source>
|
||||
<maven.compiler.target>17</maven.compiler.target>
|
||||
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
|
||||
</properties>
|
||||
|
||||
</project>
|
||||
@@ -0,0 +1,53 @@
|
||||
/*
|
||||
* This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
|
||||
*
|
||||
* The MIT License
|
||||
* Copyright © 2014-2022 Ilkka Seppälä
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
* of this software and associated documentation files (the "Software"), to deal
|
||||
* in the Software without restriction, including without limitation the rights
|
||||
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the Software is
|
||||
* furnished to do so, subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in
|
||||
* all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
* THE SOFTWARE.
|
||||
*/
|
||||
package com.iluwatar.logaggregation;
|
||||
|
||||
/**
|
||||
* The main application class responsible for demonstrating the log aggregation mechanism. Creates
|
||||
* services, generates logs, aggregates, and finally displays the logs.
|
||||
*/
|
||||
public class App {
|
||||
|
||||
/**
|
||||
* The entry point of the application.
|
||||
*
|
||||
* @param args Command line arguments.
|
||||
* @throws InterruptedException If any thread has interrupted the current thread.
|
||||
*/
|
||||
public static void main(String[] args) throws InterruptedException {
|
||||
final CentralLogStore centralLogStore = new CentralLogStore();
|
||||
final LogAggregator aggregator = new LogAggregator(centralLogStore, LogLevel.INFO);
|
||||
|
||||
final LogProducer serviceA = new LogProducer("ServiceA", aggregator);
|
||||
final LogProducer serviceB = new LogProducer("ServiceB", aggregator);
|
||||
|
||||
serviceA.generateLog(LogLevel.INFO, "This is an INFO log from ServiceA");
|
||||
serviceB.generateLog(LogLevel.ERROR, "This is an ERROR log from ServiceB");
|
||||
serviceA.generateLog(LogLevel.DEBUG, "This is a DEBUG log from ServiceA");
|
||||
|
||||
aggregator.stop();
|
||||
centralLogStore.displayLogs();
|
||||
}
|
||||
}
|
||||
+63
@@ -0,0 +1,63 @@
|
||||
/*
|
||||
* This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
|
||||
*
|
||||
* The MIT License
|
||||
* Copyright © 2014-2022 Ilkka Seppälä
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
* of this software and associated documentation files (the "Software"), to deal
|
||||
* in the Software without restriction, including without limitation the rights
|
||||
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the Software is
|
||||
* furnished to do so, subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in
|
||||
* all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
* THE SOFTWARE.
|
||||
*/
|
||||
package com.iluwatar.logaggregation;
|
||||
|
||||
import java.util.concurrent.ConcurrentLinkedQueue;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
/**
|
||||
* A centralized store for logs. It collects logs from various services and stores them.
|
||||
* This class is thread-safe, ensuring that logs from different services are safely stored
|
||||
* concurrently without data races.
|
||||
*/
|
||||
@Slf4j
|
||||
public class CentralLogStore {
|
||||
|
||||
private final ConcurrentLinkedQueue<LogEntry> logs = new ConcurrentLinkedQueue<>();
|
||||
|
||||
/**
|
||||
* Stores the given log entry into the central log store.
|
||||
*
|
||||
* @param logEntry The log entry to store.
|
||||
*/
|
||||
public void storeLog(LogEntry logEntry) {
|
||||
if (logEntry == null) {
|
||||
LOGGER.error("Received null log entry. Skipping.");
|
||||
return;
|
||||
}
|
||||
logs.offer(logEntry);
|
||||
}
|
||||
|
||||
/**
|
||||
* Displays all logs currently stored in the central log store.
|
||||
*/
|
||||
public void displayLogs() {
|
||||
LOGGER.info("----- Centralized Logs -----");
|
||||
for (LogEntry logEntry : logs) {
|
||||
LOGGER.info(
|
||||
logEntry.getTimestamp() + " [" + logEntry.getLevel() + "] " + logEntry.getMessage());
|
||||
}
|
||||
}
|
||||
}
|
||||
+120
@@ -0,0 +1,120 @@
|
||||
/*
|
||||
* This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
|
||||
*
|
||||
* The MIT License
|
||||
* Copyright © 2014-2022 Ilkka Seppälä
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
* of this software and associated documentation files (the "Software"), to deal
|
||||
* in the Software without restriction, including without limitation the rights
|
||||
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the Software is
|
||||
* furnished to do so, subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in
|
||||
* all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
* THE SOFTWARE.
|
||||
*/
|
||||
package com.iluwatar.logaggregation;
|
||||
|
||||
import java.util.concurrent.ConcurrentLinkedQueue;
|
||||
import java.util.concurrent.ExecutorService;
|
||||
import java.util.concurrent.Executors;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
/**
|
||||
* Responsible for collecting and buffering logs from different services.
|
||||
* Once the logs reach a certain threshold or after a certain time interval,
|
||||
* they are flushed to the central log store. This class ensures logs are collected
|
||||
* and processed asynchronously and efficiently, providing both an immediate collection
|
||||
* and periodic flushing.
|
||||
*/
|
||||
@Slf4j
|
||||
public class LogAggregator {
|
||||
|
||||
private static final int BUFFER_THRESHOLD = 3;
|
||||
private final CentralLogStore centralLogStore;
|
||||
private final ConcurrentLinkedQueue<LogEntry> buffer = new ConcurrentLinkedQueue<>();
|
||||
private final LogLevel minLogLevel;
|
||||
private final ExecutorService executorService = Executors.newSingleThreadExecutor();
|
||||
private final AtomicInteger logCount = new AtomicInteger(0);
|
||||
|
||||
/**
|
||||
* constructor of LogAggregator.
|
||||
*
|
||||
* @param centralLogStore central log store implement
|
||||
* @param minLogLevel min log level to store log
|
||||
*/
|
||||
public LogAggregator(CentralLogStore centralLogStore, LogLevel minLogLevel) {
|
||||
this.centralLogStore = centralLogStore;
|
||||
this.minLogLevel = minLogLevel;
|
||||
startBufferFlusher();
|
||||
}
|
||||
|
||||
/**
|
||||
* Collects a given log entry, and filters it by the defined log level.
|
||||
*
|
||||
* @param logEntry The log entry to collect.
|
||||
*/
|
||||
public void collectLog(LogEntry logEntry) {
|
||||
if (logEntry.getLevel() == null || minLogLevel == null) {
|
||||
LOGGER.warn("Log level or threshold level is null. Skipping.");
|
||||
return;
|
||||
}
|
||||
|
||||
if (logEntry.getLevel().compareTo(minLogLevel) < 0) {
|
||||
LOGGER.debug("Log level below threshold. Skipping.");
|
||||
return;
|
||||
}
|
||||
|
||||
buffer.offer(logEntry);
|
||||
|
||||
if (logCount.incrementAndGet() >= BUFFER_THRESHOLD) {
|
||||
flushBuffer();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Stops the log aggregator service and flushes any remaining logs to
|
||||
* the central log store.
|
||||
*
|
||||
* @throws InterruptedException If any thread has interrupted the current thread.
|
||||
*/
|
||||
public void stop() throws InterruptedException {
|
||||
executorService.shutdownNow();
|
||||
if (!executorService.awaitTermination(10, TimeUnit.SECONDS)) {
|
||||
LOGGER.error("Log aggregator did not terminate.");
|
||||
}
|
||||
flushBuffer();
|
||||
}
|
||||
|
||||
private void flushBuffer() {
|
||||
LogEntry logEntry;
|
||||
while ((logEntry = buffer.poll()) != null) {
|
||||
centralLogStore.storeLog(logEntry);
|
||||
logCount.decrementAndGet();
|
||||
}
|
||||
}
|
||||
|
||||
private void startBufferFlusher() {
|
||||
executorService.execute(() -> {
|
||||
while (!Thread.currentThread().isInterrupted()) {
|
||||
try {
|
||||
Thread.sleep(5000); // Flush every 5 seconds.
|
||||
flushBuffer();
|
||||
} catch (InterruptedException e) {
|
||||
Thread.currentThread().interrupt();
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
/*
|
||||
* This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
|
||||
*
|
||||
* The MIT License
|
||||
* Copyright © 2014-2022 Ilkka Seppälä
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
* of this software and associated documentation files (the "Software"), to deal
|
||||
* in the Software without restriction, including without limitation the rights
|
||||
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the Software is
|
||||
* furnished to do so, subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in
|
||||
* all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
* THE SOFTWARE.
|
||||
*/
|
||||
package com.iluwatar.logaggregation;
|
||||
|
||||
import java.time.LocalDateTime;
|
||||
import lombok.AllArgsConstructor;
|
||||
import lombok.Data;
|
||||
|
||||
/**
|
||||
* Represents a single log entry, capturing essential details like the service name,
|
||||
* log level, message, and the timestamp when the log was generated.
|
||||
*/
|
||||
@Data
|
||||
@AllArgsConstructor
|
||||
public class LogEntry {
|
||||
private String serviceName;
|
||||
private LogLevel level;
|
||||
private String message;
|
||||
private LocalDateTime timestamp;
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
/*
|
||||
* This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
|
||||
*
|
||||
* The MIT License
|
||||
* Copyright © 2014-2022 Ilkka Seppälä
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
* of this software and associated documentation files (the "Software"), to deal
|
||||
* in the Software without restriction, including without limitation the rights
|
||||
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the Software is
|
||||
* furnished to do so, subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in
|
||||
* all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
* THE SOFTWARE.
|
||||
*/
|
||||
package com.iluwatar.logaggregation;
|
||||
|
||||
/**
|
||||
* Enum representing different log levels.
|
||||
* Defines the severity of a log message, helping in filtering and prioritization.
|
||||
* <ul>
|
||||
* <li>DEBUG: Detailed information, typically of interest only when diagnosing problems.</li>
|
||||
* <li>INFO: Confirmation that things are working as expected.</li>
|
||||
* <li>ERROR: Indicates a problem that needs attention.</li>
|
||||
* </ul>
|
||||
*/
|
||||
public enum LogLevel {
|
||||
DEBUG, INFO, ERROR
|
||||
}
|
||||
+54
@@ -0,0 +1,54 @@
|
||||
/*
|
||||
* This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
|
||||
*
|
||||
* The MIT License
|
||||
* Copyright © 2014-2022 Ilkka Seppälä
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
* of this software and associated documentation files (the "Software"), to deal
|
||||
* in the Software without restriction, including without limitation the rights
|
||||
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the Software is
|
||||
* furnished to do so, subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in
|
||||
* all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
* THE SOFTWARE.
|
||||
*/
|
||||
package com.iluwatar.logaggregation;
|
||||
|
||||
import java.time.LocalDateTime;
|
||||
import lombok.AllArgsConstructor;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
/**
|
||||
* Represents a service that produces logs.
|
||||
* The logs are generated based on certain activities or events within the service.
|
||||
* Once a log is generated, it's passed on to the aggregator for further processing.
|
||||
*/
|
||||
@AllArgsConstructor
|
||||
@Slf4j
|
||||
public class LogProducer {
|
||||
|
||||
private String serviceName;
|
||||
private LogAggregator aggregator;
|
||||
|
||||
/**
|
||||
* Generates a log entry with the given log level and message.
|
||||
*
|
||||
* @param level The level of the log.
|
||||
* @param message The message of the log.
|
||||
*/
|
||||
public void generateLog(LogLevel level, String message) {
|
||||
final LogEntry logEntry = new LogEntry(serviceName, level, message, LocalDateTime.now());
|
||||
LOGGER.info("Producing log: " + logEntry.getMessage());
|
||||
aggregator.collectLog(logEntry);
|
||||
}
|
||||
}
|
||||
+80
@@ -0,0 +1,80 @@
|
||||
/*
|
||||
* This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt).
|
||||
*
|
||||
* The MIT License
|
||||
* Copyright © 2014-2022 Ilkka Seppälä
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
* of this software and associated documentation files (the "Software"), to deal
|
||||
* in the Software without restriction, including without limitation the rights
|
||||
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
* copies of the Software, and to permit persons to whom the Software is
|
||||
* furnished to do so, subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in
|
||||
* all copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
* THE SOFTWARE.
|
||||
*/
|
||||
package com.iluwatar.logaggregation;
|
||||
|
||||
import static org.mockito.ArgumentMatchers.any;
|
||||
import static org.mockito.Mockito.times;
|
||||
import static org.mockito.Mockito.verify;
|
||||
|
||||
import java.time.LocalDateTime;
|
||||
import org.junit.jupiter.api.BeforeEach;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.extension.ExtendWith;
|
||||
import org.mockito.Mock;
|
||||
import org.mockito.junit.jupiter.MockitoExtension;
|
||||
|
||||
@ExtendWith(MockitoExtension.class)
|
||||
class LogAggregatorTest {
|
||||
|
||||
@Mock
|
||||
private CentralLogStore centralLogStore;
|
||||
private LogAggregator logAggregator;
|
||||
|
||||
@BeforeEach
|
||||
void setUp() {
|
||||
logAggregator = new LogAggregator(centralLogStore, LogLevel.INFO);
|
||||
}
|
||||
|
||||
@Test
|
||||
void whenThreeInfoLogsAreCollected_thenCentralLogStoreShouldStoreAllOfThem() {
|
||||
logAggregator.collectLog(createLogEntry(LogLevel.INFO, "Sample log message 1"));
|
||||
logAggregator.collectLog(createLogEntry(LogLevel.INFO, "Sample log message 2"));
|
||||
|
||||
verifyNoInteractionsWithCentralLogStore();
|
||||
|
||||
logAggregator.collectLog(createLogEntry(LogLevel.INFO, "Sample log message 3"));
|
||||
|
||||
verifyCentralLogStoreInvokedTimes(3);
|
||||
}
|
||||
|
||||
@Test
|
||||
void whenDebugLogIsCollected_thenNoLogsShouldBeStored() {
|
||||
logAggregator.collectLog(createLogEntry(LogLevel.DEBUG, "Sample debug log message"));
|
||||
|
||||
verifyNoInteractionsWithCentralLogStore();
|
||||
}
|
||||
|
||||
private static LogEntry createLogEntry(LogLevel logLevel, String message) {
|
||||
return new LogEntry("ServiceA", logLevel, message, LocalDateTime.now());
|
||||
}
|
||||
|
||||
private void verifyNoInteractionsWithCentralLogStore() {
|
||||
verify(centralLogStore, times(0)).storeLog(any());
|
||||
}
|
||||
|
||||
private void verifyCentralLogStoreInvokedTimes(int times) {
|
||||
verify(centralLogStore, times(times)).storeLog(any());
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user