skywalking docker部署报错Out of Memory Error (workerManager.hpp:70)
  BW1XAG7RjrKC 2023年12月08日 43 0


部分报错内容

[0.003s][warning][os,thread] Failed to start thread "GC Thread#0" - pthread_create failed (EPERM) for attributes: stacksize: 1024k, guardsize: 4k, detached.
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create worker GC thread. Out of system resources.
# Can not save log file, dump to screen..
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create worker GC thread. Out of system resources.
# Possible reasons:
#   The system is out of physical RAM or swap space
#   The process is running with CompressedOops enabled, and the Java Heap may be blocking the growth of the native heap
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
#   JVM is running with Unscaled Compressed Oops mode in which the Java heap is
#     placed in the first 4GB address space. The Java Heap base address is the
#     maximum limit for the native heap growth. Please use -XX:HeapBaseMinAddress
#     to set the Java Heap base and to place the Java Heap above 4GB virtual address.
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (workerManager.hpp:70), pid=1, tid=1
#
# JRE version:  (11.0.16.1+1) (build )
# Java VM: OpenJDK 64-Bit Server VM (11.0.16.1+1, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64)
# Core dump will be written. Default location: Core dumps may be processed with "/usr/libexec/abrt-hook-ccpp %s %c %p %u %g %t e %P %I %h" (or dumping to /skywalking/core.1)
#

---------------  T H R E A D  ---------------

Current thread (0x000055bf39a30000):  JavaThread "Unknown thread" [_thread_in_vm, id=1, stack(0x00007ffd91f2c000,0x00007ffd9272b000)]

Stack: [0x00007ffd91f2c000,0x00007ffd9272b000],  sp=0x00007ffd92725500,  free space=8165k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0xf4720a]  VMError::report_and_die(int, char const*, char const*, __va_list_tag*, Thread*, unsigned char*, void*, void*, char const*, int, unsigned long)+0x1ca
V  [libjvm.so+0xf480cb]  VMError::report_and_die(Thread*, char const*, int, unsigned long, VMErrorType, char const*, __va_list_tag*)+0x2b^C
V  [libjvm.so+0x6d3df5]  report_vm_out_of_memory(char const*, int, unsigned long, VMErrorType, char const*, ...)+0xd5
V  [libjvm.so+0xf7466b]  AbstractWorkGang::add_workers(unsigned int, bool)+0x14b
V  [libjvm.so+0x7afb55]  G1CollectedHeap::G1CollectedHeap(G1CollectorPolicy*)+0x5b5
V  [libjvm.so+0x7a1649]  G1Arguments::create_heap()+0x69
V  [libjvm.so+0xf02951]  Universe::initialize_heap()+0x31
V  [libjvm.so+0xf02c06]  universe_init()+0x46
V  [libjvm.so+0x8a9497]  init_globals()+0x57
V  [libjvm.so+0xeda277]  Threads::create_vm(JavaVMInitArgs*, bool*)+0x357
V  [libjvm.so+0x9670d2]  JNI_CreateJavaVM+0x52
C  [libjli.so+0x49af]  JavaMain+0x8f
C  [libjli.so+0x9d94]  CallJavaMainInNewThread+0x74
C  [libjli.so+0x637d]  ContinueInNewThread+0x4d
C  [libjli.so+0x7a00]  JLI_Launch+0x1470
C  [java+0xa73]

---------------  P R O C E S S  ---------------

Threads class SMR info:
_java_thread_list=0x000055949a8b38d0, length=0, elements={
}

Java Threads: ( => current thread )

Other Threads:

=>0x000055949a8ec000 (exited) JavaThread "Unknown thread" [_thread_in_vm, id=1, stack(0x00007fff126b6000,0x00007fff12eb5000)]

Threads with active compile tasks:

VM state:not at safepoint (not fully initialized)

VM Mutex/Monitor currently owned by a thread: None

[error occurred during error reporting (printing compressed oops mode), id 0xb, SIGSEGV (0xb) at pc=0x00007f0c62701283]

GC Heap History (0 events):
No events

Deoptimization events (0 events):
No events

Classes redefined (0 events):
No events

Internal exceptions (0 events):
No events

Events (2 events):
Event: 0.002 Protecting memory [0x00007fff126b6000,0x00007fff126ba000] with protection modes 0
Event: 0.002 Loaded shared library /opt/java/openjdk/lib/libzip.so

解决

docker 版本低,升级docker版本

参考

Cannot create worker GC thread. Out of system resources. · Issue #3020 · adoptium/temurin-build · GitHub

docker - Container Fails to Start: Insufficient memory for the Java Runtime Environment to continue - Stack Overflowy

原因

I see that your Docker image uses Ubuntu 22.04 LTS as its base. Recently base Java images were rebuilt on top of this LTS version, which caused a lot of issues on older Docker runtimes. Most likely this is what you're experiencing. It has nothing to do with memory, but rather with Docker incompatibility with a newer Linux version used as a base image.

Your operational server has Docker server version 20.10.10, while the failing server has version 20.10.09. The incompatibility issue was fixed exactly in Docker 20.10.10. Some more technical details on the incompatibility issue are available here.

The solution would be to upgrade the failing server to at least Docker 20.10.10.

翻译:

 我看到你的Docker镜像使用Ubuntu 22.04 LTS作为基础。最近,在这个LTS版本的基础上重新构建了基本Java映像,这在旧的Docker运行时上造成了很多问题。很可能这就是你正在经历的。它与内存无关,而是与Docker与用作基础映像的较新Linux版本的不兼容有关。

您的操作服务器的Docker服务器版本为20.10.10,而故障服务器的版本为20.10.09。Docker 20.10.10中准确地修复了不兼容问题。有关不兼容问题的更多技术细节,请访问此处。

解决方案是将出现故障的服务器至少升级到Docker 20.10.10。

【版权声明】本文内容来自摩杜云社区用户原创、第三方投稿、转载,内容版权归原作者所有。本网站的目的在于传递更多信息,不拥有版权,亦不承担相应法律责任。如果您发现本社区中有涉嫌抄袭的内容,欢迎发送邮件进行举报,并提供相关证据,一经查实,本社区将立刻删除涉嫌侵权内容,举报邮箱: cloudbbs@moduyun.com

  1. 分享:
最后一次编辑于 2023年12月08日 0

暂无评论

推荐阅读
  xaeiTka4h8LY   2024年05月17日   48   0   0 数据库JavaSQL
BW1XAG7RjrKC