Using "grep" with "kubectl logs" to Filter Container/Pod Logs

 kubectl logs command helps us to inspect logs of pods in AKS/kubernetes and useful to diagnose issues. However, when there is too much logs it is harder to read through and find out errors easily. Further, filtering out logs for a given timestamp may be useful at times to identify the issues. In this post let's explore usage of grep with kubectl logs command to filter logs. 

Let's take first example to filter for a timestamp in keda operator pod logs. Here -i says to ignore case in logs.

kubectl logs keda-operator-79d756dd66-69gsc -n keda | grep -i '2025-03-04T07:20:24'

Reducing Log Analytics Cost for App Insights by Removing Successful Dependency Logs Ingestion from .NET Applications

 We have discussed how to reduce log analytics cost by reducing container log ingestion from apps deployed to AKS in the post "Reducing Log Analytics Cost by Preventing Container Logs Ingession from Azure Kubernetes Services (AKS)". However, when we inspected the charts o f data ingestion after reducing container logs, the next major data volume is ingested by app dependecy logs, coming from .NET apps running in AKS cluster. App dependecy logs are mostly information logs coming from the .NET SDK and other dependecies such as Azure storage etc. These logs are useful when there is an issue or error in the dependecy calls. However, when the dependency has run with successful state, there is not much use of that information. Since, the highest cost for app insights log analytics cost is incurred by dependency logs after we have removed container logs, it is worth to reduce the cost of log analytics data ingestion for app insights, by removing the successful dependecy logs from the app insights.

The expected outcome is as shown below. After the removal of succesful dpenedecy logs data ingestion for dependecy logs has reduce to almost zero.

Reducing Log Analytics Cost by Preventing Container Logs Ingestion from Azure Kubernetes Services (AKS)

 Monitoring is an essential part of a deployment of software on a platform such as AKS. However, once monitoring enabled there could be significant cost involved for monitoring data. When we enable log analytics workspace to ingest monitoring data from AKS, by default AKS will ingest all container logs, except for kube-system and gatekeeper-system namespaces. If our application are having large amout of container logs generated, then the cost will be lot higher for log analytics workspace. In case we are using app insights/ or an alternative system to monitor the application logs, for the applications deployed to AKS, we have enuough information to diagnose issues etc. Therefore, to reduce unnecessary cost log analytics can be done, by preventing AKS ingestion of the container logs to log analytics workspace. Let's explore the steps.

Expected outcome

As shown in the below figure, the log analytics workspace ingestion over time chart, is indicating the container logs data is no longer getting ingested after the change is applied.

 

View More lines in VS Code Integrated Terminal

VS Code might be your favourite development tool to develop in any language in any platform. It has integrated terminals, for PowerShell, comand prompt, Git Bash, WSL  Azure Cloud shell etc. You may find missing teminal logs specially when you run commands producing larger logs for review such as terraform plan or bicep with --what-if. By default integrated terminal in VS Code only displays last 1000 lines. Let's look at how we can increase the number of log lines in the VS code integrated terminal.

Log All Requests in ASP.NET Core API

Logging all incoming requests and the respose returned data in an API are useful way to diagnose any issues and monitor requests to an API. You can easily enable request logging in AP.NET Core 6 APIs, by setting the log level in app settings. Enabling such request reponse data logging will be useful in containerized APIs deployed to Kubernetes or even for APIs deployed to AzureApp Services  to diagnose and monitoring purposes. Let's look at how to enable request logging.