SpringBoot操作Docker实现多语言代码沙箱, 并用容器池优化性能
本文可能很长, 但能保证能让你从0-1实现一个高可用、高性能、高扩展性的多语言代码沙箱。
Docker
为什么用Docker做代码沙箱?
为了提升系统的安全性, 把程序和宿主机隔离, 使得某个程序的执行不会影响系统本身。
Docker核心概念
镜像:用来创建容器的安装包,可以理解为给电脑安装操作系统的系统镜像
容器:通过镜像来创建的一套运行环境,一个容器里可以运行多个程序,可以理解为一个电脑实例。
Dockerfile:制作镜像的文件,可以理解为制作镜像的一个清单
镜像仓库:存放镜像的仓库,用户可以从仓库下载现成的镜像,也可以把做好的镜像放到仓库里
为什么用容器池?
因为不用容器池的话,每次执行代码都要了:连接Docker-创建容器-执行代码-删除容器
,极大的浪费了带宽和内存。而用了容器池,每次就只需要:连接docker-执行代码,初始化容器在第一次就会执行,后续只有有需要了才会动态扩展,极大提高了复用性,不用每次都增加和删除了。
安装Docker Desktop
- 进入Docker官网:
https://www.docker.com - 选择自己电脑的系统, 下载并安装对应版本的Docker
- 测试,出现版本号说明安装成功了
如果是MacOS(非MacOS忽略)需要使用TCP连接Docker,Docker Desktop 默认不支持直接通过 TCP 端口访问 Docker 守护进程。不过,你可以通过运行一个socat
容器来实现这一功能。以下是具体步骤:
(1) 拉取socat镜像
docker pull alpine/socat
(2) 开放端口(如果考虑安全性, 请换成局域网地址)
export DOCKER_HOST=tcp://0.0.0.0:2375
(3) 运行socat容器
docker run -d --restart=always -p 0.0.0.0:2375:2375 -v /var/run/docker.sock:/var/run/docker.sock alpine/socat tcp-listen:2375,fork,reuseaddr,ignoreeof unix-connect:/var/run/docker.sock
(4) 测试
curl http://localhost:2375/info
(5) 出现以下结果, 说明成功了
Java操作Docker
用docker-java这个开源项目就能实现在springboot的项目中去操作docker容器了。下面是这个项目的开源地址
https://github.com/docker-java/docker-java
项目结构
以下为项目整体目录, 供参考:
核心依赖
<properties>
<java.version>17java.version>
<docker.java.version>3.3.0docker.java.version>
properties>
<dependency>
<groupId>com.github.docker-javagroupId>
<artifactId>docker-java-coreartifactId>
<version>${docker.java.version}version>
dependency>
<dependency>
<groupId>com.github.docker-javagroupId>
<artifactId>docker-java-apiartifactId>
<version>${docker.java.version}version>
dependency>
<dependency>
<groupId>com.github.docker-javagroupId>
<artifactId>docker-java-transportartifactId>
<version>${docker.java.version}version>
dependency>
<dependency>
<groupId>com.github.docker-javagroupId>
<artifactId>docker-java-transport-httpclient5artifactId>
<version>${docker.java.version}version>
dependency>
核心代码
1. 定义支持的编程语言、请求、响应和控制类
// 定义支持的编程语言
public enum Language {
JAVA("java", "Main.java", "openjdk:17-ea-10-jdk-slim"),
PYTHON("python", "main.py", "python:3.12.10-alpine3.21"),
C("c", "main.c", "gcc:10.5.0");
private final String name;
private final String fileName;
private final String image;
Language(String name, String fileName, String image) {
this.name = name;
this.fileName = fileName;
this.image = image;
}
public String getName() {
return name;
}
public String getFileName() {
return fileName;
}
public String getImage() {
return image;
}
}
请求类:
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
@Data
@AllArgsConstructor
@NoArgsConstructor
public class CodeExecuteRequest {
private String code;
private String language;
private String inputArgs;
}
响应类
import lombok.Data;
@Data
public class CodeExecuteResponse {
private String result;
private Double time;
private Long memory;
}
控制类
@Slf4j
@RestController
@RequestMapping("/codesandbox")
public class CodeSandboxController {
@Resource
private DockerCodeService dockerCodeService;
@PostMapping("/execute")
public CodeExecuteResponse executeCode(@RequestBody CodeExecuteRequest codeExecuteRequest) {
log.info("executeCodeRequest: {}", codeExecuteRequest);
if(codeExecuteRequest == null) {
throw new RuntimeException("请求参数为空");
}
return dockerCodeService.codeExecute(codeExecuteRequest);
}
}
2. 拉取代码执行环境镜像(C, Java, Python)
拉取镜像(image1-image3逐个替换, 分别执行一次)
public class DockerDownloadImage {
public static void main(String[] args) throws InterruptedException {
// 创建远程Docker服务器的配置
DockerClientConfig config = DefaultDockerClientConfig.createDefaultConfigBuilder()
.withDockerHost("tcp://192.168.134.139:2375") // 指定远程Docker服务器地址和端口
.build();
// 创建HTTP客户端
DockerHttpClient httpClient = new ApacheDockerHttpClient.Builder()
.dockerHost(config.getDockerHost())
.sslConfig(config.getSSLConfig())
.connectionTimeout(Duration.ofSeconds(30))
.responseTimeout(Duration.ofSeconds(45))
.build();
// 创建DockerClient
DockerClient dockerClient = DockerClientImpl.getInstance(config,httpClient);
// 拉取镜像(以下镜像逐个替换, 分别执行一次)
String image1 = "gcc:10.5.0";
String image2 = "openjdk:17-ea-10-jdk-slim";
String image3 = "python:3.12.10-alpine3.21";
PullImageCmd pullImageCmd = dockerClient.pullImageCmd(image);
PullImageResultCallback pullImageResultCallback = new PullImageResultCallback() {
@Override
public void onNext(PullResponseItem item) {
System.out.println("下载镜像:" + item.getStatus());
super.onNext(item);
}
};
pullImageCmd
.exec(pullImageResultCallback)
.awaitCompletion();
System.out.println("下载完成");
//创建容器
CreateContainerCmd containerCmd = dockerClient.createContainerCmd(image);
CreateContainerResponse containerResponse = containerCmd.withCmd("echo", "Hello, Docker!")
.exec();
String containerId = containerResponse.getId();
System.out.println(containerResponse);
System.out.println(containerId);
//查看容器状态
ListContainersCmd listContainersCmd = dockerClient.listContainersCmd();
List<Container> containerList = listContainersCmd.withShowAll(true).exec();
for (Container container : containerList) {
System.out.println(container);
}
//启动容器
dockerClient.startContainerCmd(containerId).exec();
//日志可能会很长,所以可以使用回调函数来处理日志输出
ResultCallbackTemplate<?, Frame> callback = new ResultCallbackTemplate<>() {
@Override
public void onNext(Frame frame) {
StreamType streamType = frame.getStreamType();
String payload = new String(frame.getPayload());
System.out.println("日志:" + streamType + ": " + payload);
}
};
//查看容器日志
dockerClient.logContainerCmd(containerId)
.withStdOut(true)
.withStdErr(true)
.exec(callback).
awaitCompletion();//阻塞等待日志输出完成
// //删除容器
// dockerClient.removeContainerCmd(containerId).exec();//.withForce强制删除
// //删除镜像
// dockerClient.removeImageCmd(image).withForce(true).exec();
}
}
3. 容器池
Docker容器池管理类, 用于:
- 初始化容器池
- 创建容器并添加到池中
- 动态获取容器
- 返回容器到池中,即在整个代码沙箱执行流程中,使用完容器后不直接销毁,而是调用此方法将其归还到池中,形成了高效的容器资源循环利用机制。
- 清理容器工作目录并销毁容器
/**
* Docker容器池管理类
*/
public class DockerContainerPool {
private static final int MIN_POOL_SIZE = 2; // 每种语言的最小容器数量
private static final int MAX_POOL_SIZE = 10; // 每种语言的最大容器数量
// 每种语言的容器池
private final Map<Language, Queue<ContainerInstance>> containerPools = new ConcurrentHashMap<>();
// 每种语言当前的容器数量
private final Map<Language, AtomicInteger> containerCounts = new ConcurrentHashMap<>();
private final DockerClient dockerClient;
public DockerContainerPool(DockerClient dockerClient) {
this.dockerClient = dockerClient;
// 初始化容器池
initializePool();
}
/**
* 初始化容器池
*/
private void initializePool() {
for (Language language : Language.values()) {
containerPools.put(language, new ConcurrentLinkedQueue<>());
containerCounts.put(language, new AtomicInteger(0));
// 预创建最小数量的容器
for (int i = 0; i < MIN_POOL_SIZE; i++) {
createAndAddContainer(language);
}
}
}
/**
* 创建容器并添加到池中
*/
private void createAndAddContainer(Language language) {
try {
// 创建容器
CreateContainerResponse container = dockerClient.createContainerCmd(language.getImage())
.withWorkingDir("/tmp")
.withMemory((long) (100 * 1024 * 1024))
.withMemorySwap(0L)
.withTty(true)
.exec();
// 启动容器
String containerId = container.getId();
dockerClient.startContainerCmd(containerId).exec();
// 将容器添加到池中
containerPools.get(language).add(new ContainerInstance(containerId, language));
containerCounts.get(language).incrementAndGet();
System.out.println("容器已创建并添加到池中: " + containerId + " (语言: " + language.getName() + ")");
} catch (Exception e) {
System.err.println("创建容器时出错: " + e.getMessage());
}
}
/**
* 获取容器
*/
public synchronized ContainerInstance getContainer(Language language) {
Queue<ContainerInstance> pool = containerPools.get(language);
// 如果池中有可用容器,直接返回
ContainerInstance container = pool.poll();
if (container != null) {
return container;
}
// 如果池中没有可用容器且未达到最大容器数量,创建新容器
int currentCount = containerCounts.get(language).get();
if (currentCount < MAX_POOL_SIZE) {
createAndAddContainer(language);
return containerPools.get(language).poll();
}
// 如果已达到最大容器数量,等待可用容器
try {
// 等待一段时间后重试
Thread.sleep(1000);
return getContainer(language);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new RuntimeException("获取容器时被中断");
}
}
/**
* 返回容器到池中
*/
public synchronized void returnContainer(ContainerInstance container) {
try {
// 清理容器工作目录
cleanupContainer(container);
// 将容器返回到池中
containerPools.get(container.getLanguage()).add(container);
} catch (Exception e) {
System.err.println("返回容器到池中出错: " + e.getMessage());
// 如果容器状态异常,销毁并创建新容器
removeContainer(container.getContainerId());
createAndAddContainer(container.getLanguage());
}
}
/**
* 清理容器工作目录
*/
private void cleanupContainer(ContainerInstance container) {
// 这里实现清理容器内工作目录的逻辑
try {
// 示例:删除工作目录下的所有文件
ExecCreateCmdResponse exec = dockerClient.execCreateCmd(container.getContainerId())
.withCmd("/bin/sh", "-c", "rm -rf /tmp/code/* && mkdir -p /tmp/code")
.exec();
dockerClient.execStartCmd(exec.getId())
.exec(new ExecStartResultCallback())
.awaitCompletion(5, TimeUnit.SECONDS);
} catch (Exception e) {
System.err.println("清理容器时出错: " + e.getMessage());
throw new RuntimeException("清理容器时出错"); // 向上传递异常
}
}
/**
* 移除并销毁容器
*/
private void removeContainer(String containerId) {
try {
dockerClient.stopContainerCmd(containerId).withTimeout(5).exec();
dockerClient.removeContainerCmd(containerId).withForce(true).exec();
System.out.println("容器已销毁: " + containerId);
} catch (Exception e) {
System.err.println("销毁容器时出错: " + e.getMessage());
}
}
/**
* 容器实例类
*/
@Getter
public static class ContainerInstance {
private final String containerId;
private final Language language;
public ContainerInstance(String containerId, Language language) {
this.containerId = containerId;
this.language = language;
}
}
}
4. 代码执行实现类
该类用于保存基础配置,连接并初始化Docker客户端和容器池,然后执行代码,流程如下:
- 保存代码到本地临时文件
- 创建并启动容器
- 复制代码到容器
- 编译代码(如果需要)
- 运行代码并获取结果
/**
* Docker代码沙箱
*/
@Service
@Slf4j
public class DockerCodeService {
// 基础配置
private static final String TEMP_DIR = "tempCode";
private static final String DOCKER_HOST = "tcp://192.168.134.139:2375";
private static final String WORK_DIR = "/tmp/code";
private static final long TIMEOUT = 10000L; // 10秒超时
// 容器池管理
private static final DockerClient dockerClient;
private static final DockerContainerPool containerPool;
static {
// 初始化Docker客户端和容器池
dockerClient = createDockerClient();
containerPool = new DockerContainerPool(dockerClient);
}
public CodeExecuteResponse codeExecute(CodeExecuteRequest codeExecuteRequest) {
// 选择要运行的语言
String code = codeExecuteRequest.getCode();
String inputArgs = codeExecuteRequest.getInputArgs();
//获取语言类型
Language language = getLanguage(codeExecuteRequest.getLanguage());
// 从容器池获取容器实例
ContainerInstance containerInstance = null;
String containerId = null;
try {
// 1. 保存代码到本地临时文件
String codePath = saveCodeToFile(code, language);
// 2. 创建并启动容器
containerInstance = containerPool.getContainer(language);
containerId = containerInstance.getContainerId();
// 3. 复制代码到容器
copyCodeToContainer(dockerClient, containerId, codePath);
// 4. 编译代码(如果需要)
long startTime = System.currentTimeMillis();
if (language == Language.JAVA || language == Language.C) {
compileCode(dockerClient, containerId, language);
}
// 5. 运行代码并获取结果
String codeExecuteResponse = runCode(dockerClient, containerId, inputArgs, language);
CodeExecuteResponse result = new CodeExecuteResponse();
result.setResult(codeExecuteResponse);
result.setTime((System.currentTimeMillis()-startTime)/1000.0);//计算执行时间
result.setMemory(0L);
return result;
} catch (Exception e) {
System.err.println("执行过程中发生错误: " + e.getMessage());
throw new RuntimeException("执行过程中发生错误: " + e.getMessage());
} finally {
// 6. 清理临时文件
if(containerInstance != null) {
containerPool.returnContainer(containerInstance);//不销毁容器, 而是将其返回到容器池
}
}
}
// 获取语言对象
private static Language getLanguage(String requestLanguage) {
if (requestLanguage.equals("java")) {
return Language.JAVA;
} else if (requestLanguage.equals("python")) {
return Language.PYTHON;
} else if (requestLanguage.equals("c")) {
return Language.C;
} else {
System.out.println("不支持的语言类型");
throw new RuntimeException("不支持的语言类型");
}
}
/**
* 创建Docker客户端
*/
private static DockerClient createDockerClient() {
// 配置Docker连接
DockerClientConfig config = DefaultDockerClientConfig.createDefaultConfigBuilder()
.withDockerHost(DOCKER_HOST)
.withApiVersion("1.41")
.build();
// 创建HTTP客户端
DockerHttpClient httpClient = new ApacheDockerHttpClient.Builder()
.dockerHost(config.getDockerHost())
.sslConfig(config.getSSLConfig())
.connectionTimeout(Duration.ofSeconds(30))
.responseTimeout(Duration.ofSeconds(30))
.maxConnections(100)
.build();
// 返回Docker客户端
return DockerClientImpl.getInstance(config, httpClient);
}
/**
* 保存代码到本地临时文件
*/
private static String saveCodeToFile(String code, Language language) {
// 创建临时目录
String userDir = System.getProperty("user.dir");
String tempDir = userDir + File.separator + TEMP_DIR;
String codePath = tempDir + File.separator + UUID.randomUUID();
// 确保目录存在
FileUtil.mkdir(codePath);
// 写入代码文件
String filePath = codePath + File.separator + language.getFileName();
FileUtil.writeString(code, filePath, StandardCharsets.UTF_8);
System.out.println("代码已保存到: " + filePath);
return codePath;
}
/**
* 创建并启动容器
*/
private static String createContainer(DockerClient docker, Language language) throws InterruptedException {
// 创建容器
CreateContainerResponse container = docker.createContainerCmd(language.getImage())
.withWorkingDir("/tmp")
.withMemory((long) (100 *1024 * 1024))//不高于100MB
.withMemorySwap(0L)//禁止内存到硬盘交换
.withTty(true)
.exec();
String containerId = container.getId();
// 启动容器
docker.startContainerCmd(containerId).exec();
// 等待容器启动
Thread.sleep(500);
System.out.println("容器已创建并启动: " + containerId + " (镜像: " + language.getImage() + ")");
return containerId;
}
/**
* 复制代码到容器
*/
private static void copyCodeToContainer(DockerClient docker, String containerId, String codePath) throws InterruptedException {
// 创建工作目录
execCommand(docker, containerId, "mkdir -p " + WORK_DIR);
// 读取代码文件
File dir = new File(codePath);
File[] files = dir.listFiles();
if (files != null) {
for (File file : files) {
if (file.isFile()) {
// 读取文件内容
String content = FileUtil.readString(file, StandardCharsets.UTF_8);
String fileName = file.getName();
// Base64编码避免特殊字符问题
String base64Content = Base64.getEncoder().encodeToString(content.getBytes(StandardCharsets.UTF_8));
// 复制到容器
String command = "echo " + base64Content + " | base64 -d > " + WORK_DIR + "/" + fileName;
execCommand(docker, containerId, command);
System.out.println("文件 " + fileName + " 已复制到容器");
}
}
}
}
/**
* 编译代码(如果需要)
*/
private static void compileCode(DockerClient docker, String containerId, Language language) throws Exception {
if (language == Language.JAVA) {
System.out.println("开始编译Java代码...");
// 执行编译
String command = "cd " + WORK_DIR + " && javac " + language.getFileName();
execCommandWithOutput(docker, containerId, command);
// 检查是否编译成功
command = "find " + WORK_DIR + " -name "*.class"";
String classFiles = execCommandWithOutput(docker, containerId, command);
if (classFiles.trim().isEmpty()) {
throw new Exception("编译失败,未生成class文件");
}
System.out.println("编译成功,生成文件: " + classFiles);
} else if (language == Language.PYTHON) {
// Python不需要编译,但可以进行语法检查
System.out.println("检查Python语法...");
String command = "cd " + WORK_DIR + " && python -m py_compile " + language.getFileName();
execCommandWithOutput(docker, containerId, command);
System.out.println("Python语法检查完成");
} else if (language == Language.C) {
System.out.println("开始编译C代码...");
// 编译C代码,输出为可执行文件program
String command = "cd " + WORK_DIR + " && gcc -o program " + language.getFileName();
execCommandWithOutput(docker, containerId, command);
// 检查是否生成了可执行文件
command = "ls -la " + WORK_DIR + "/program";
String execFile = execCommandWithOutput(docker, containerId, command);
if (execFile.trim().isEmpty() || execFile.contains("No such file")) {
throw new Exception("编译失败,未生成可执行文件");
}
System.out.println("C代码编译成功,生成可执行文件: program");
}
}
/**
* 运行代码并获取结果
*/
private static String runCode(DockerClient docker, String containerId, String inputArgs, Language language) throws InterruptedException {
System.out.println("开始执行代码...");
String command;
if (language == Language.JAVA) {
// Java执行命令
String className = language.getFileName().substring(0, language.getFileName().lastIndexOf('.'));
command = "cd " + WORK_DIR + " && echo "" + inputArgs + "" | java " + className;
} else if (language == Language.PYTHON) {
// Python执行命令
command = "cd " + WORK_DIR + " && echo "" + inputArgs + "" | python " + language.getFileName();
} else {
// C程序执行命令
command = "cd " + WORK_DIR + " && echo "" + inputArgs + "" | ./program";
}
System.out.println("执行命令: " + command);
return execCommandWithOutput(docker, containerId, command);
}
/**
* 执行命令(无需返回结果)
*/
private static void execCommand(DockerClient docker, String containerId, String command) throws InterruptedException {
ExecCreateCmdResponse exec = docker.execCreateCmd(containerId)
.withCmd("/bin/sh", "-c", command)
.exec();
docker.execStartCmd(exec.getId())
.exec(new ExecStartResultCallback())
.awaitCompletion(TIMEOUT, TimeUnit.MILLISECONDS);
}
/**
* 执行命令并返回输出结果
*/
private static String execCommandWithOutput(DockerClient docker, String containerId, String command) throws InterruptedException {
//
ExecCreateCmdResponse exec = docker.execCreateCmd(containerId)
.withCmd("/bin/sh", "-c", command)
.withAttachStdout(true)
.withAttachStderr(true)
.exec();
StringBuilder output = new StringBuilder();
StringBuilder error = new StringBuilder();
docker.execStartCmd(exec.getId())
.exec(new ExecStartResultCallback() {
@Override
public void onNext(Frame frame) {
try {
if (frame.getStreamType() == StreamType.STDOUT) {
output.append(new String(frame.getPayload()));
} else if (frame.getStreamType() == StreamType.STDERR) {
error.append(new String(frame.getPayload()));
}
} catch (Exception e) {
System.err.println("处理输出时出错: " + e.getMessage());
}
super.onNext(frame);
}
})
.awaitCompletion(TIMEOUT, TimeUnit.MILLISECONDS);//阻塞等待命令执行完成, 配置超时时间
if (error.length() > 0) {
System.err.println("执行时出现错误: " + error);
}
final long[] memory = {0};
try {
StatsCmd statsCmd = docker.statsCmd(containerId); // 使用容器ID
statsCmd.exec(new ResultCallback<Statistics>() {
@Override
public void onNext(Statistics statistics) {
if (statistics.getMemoryStats().getUsage() != null) {
memory[0] =statistics.getMemoryStats().getUsage();
}
}
@Override
public void close() {}
@Override
public void onStart(Closeable closeable) {}
@Override
public void onError(Throwable throwable) {}
@Override
public void onComplete() {}
});
// 等待足够时间收集统计信息
Thread.sleep(500);
// 手动关闭统计收集
statsCmd.close();
} catch(Exception e) {
System.err.println("获取内存统计信息时出错: " + e.getMessage());
}
System.out.println("执行结果:" + output.toString());
return output.toString();
}
}
测试
本文地址:https://www.vps345.com/15129.html