minio对象存储搭建与使用
  oIa1edJoFmXP 2023年12月09日 29 0

minio对象存储搭建与使用(2023-12-08)

官网:https://min.io/

安装教程:https://min.io/docs/minio/linux/operations/installation.html

一、直接下载使用

wget https://dl.min.io/server/minio/release/linux-amd64/minio
chmod +x minio
MINIO_ROOT_USER=admin MINIO_ROOT_PASSWORD=password ./minio server /mnt/data --console-address ":9001"

下载minio,给与执行权限,赋值账号密码启动minio server,同时指定minioServer的存储地址, api接口端口默认为9000,console的端口默认是随机,可以通过--console-address=":9001"设置为指定的9001端口。

minio对象存储搭建与使用_html


二、配置Service+域名使用

下载minio,mc工具

下载minio,mc工具,给与可执行权限,放到/usr/local/bin/目录下。

wget https://dl.min.io/server/minio/release/linux-amd64/minio
chmod +x minio
#MINIO_ROOT_USER=admin MINIO_ROOT_PASSWORD=password ./minio server /mnt/data --console-address ":9001"

wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
#mc alias set myminio/ http://MINIO-SERVER MYUSER MYPASSWORD

#mv minio /usr/bin/
#mv mc /usr/bin/
mv minio /usr/local/bin/
mv mc /usr/local/bin/

https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-minio-single-node-single-drive.html#

创建systemd的Service文件

创建minio.service文件于/usr/lib/systemd/system/minio.service。

[Unit]
Description=MinIO
Documentation=https://min.io/docs/minio/linux/index.html
Wants=network-online.target
After=network-online.target
AssertFileIsExecutable=/usr/local/bin/minio

[Service]
WorkingDirectory=/usr/local

User=minio-user
Group=minio-user
ProtectProc=invisible

EnvironmentFile=-/etc/default/minio
ExecStartPre=/bin/bash -c "if [ -z \"${MINIO_VOLUMES}\" ]; then echo \"Variable MINIO_VOLUMES not set in /etc/default/minio\"; exit 1; fi"
ExecStart=/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES

# MinIO RELEASE.2023-05-04T21-44-30Z adds support for Type=notify (https://www.freedesktop.org/software/systemd/man/systemd.service.html#Type=)
# This may improve systemctl setups where other services use `After=minio.server`
# Uncomment the line to enable the functionality
# Type=notify

# Let systemd restart this service always
Restart=always

# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=65536

# Specifies the maximum number of threads this process can create
TasksMax=infinity

# Disable timeout logic and wait until process is stopped
TimeoutStopSec=infinity
SendSIGKILL=no

[Install]
WantedBy=multi-user.target

# Built for ${project.name}-${project.version} (${project.name})

从上面配置文件可以看到还需要创建运行的用户和用户组,还需要Env环境文件。


创建用户和用户组

groupadd -r minio-user
useradd -M -r -g minio-user minio-user
chown minio-user:minio-user /mnt/disk1 /mnt/disk2 /mnt/disk3 /mnt/disk4

后面的/mnt/disk1等为minio Server的存储目录,如果是配置一个就给一个入 /mnt/data。


创建Env环境文件

# MINIO_ROOT_USER and MINIO_ROOT_PASSWORD sets the root account for the MinIO server.
# This user has unrestricted permissions to perform S3 and administrative API operations on any resource in the deployment.
# Omit to use the default values 'minioadmin:minioadmin'.
# MinIO recommends setting non-default values as a best practice, regardless of environment

MINIO_ROOT_USER=myminioadmin
MINIO_ROOT_PASSWORD=minio-secret-key-change-me

# MINIO_VOLUMES sets the storage volume or path to use for the MinIO server.

MINIO_VOLUMES="/mnt/data"

# MINIO_SERVER_URL sets the hostname of the local machine for use with the MinIO Server
# MinIO assumes your network control plane can correctly resolve this hostname to the local machine

# Uncomment the following line and replace the value with the correct hostname for the local machine and port for the MinIO server (9000 by default).

#MINIO_SERVER_URL="http://minio.example.net:9000"


启动minio服务

systemctl start minio.service
systemctl status minio.service
journalctl -f -u minio.service
systemctl enable minio.service

minio对象存储搭建与使用_linux_02

minio对象存储搭建与使用_html_03


安装和配置Nginx

https://min.io/docs/minio/linux/integrations/setup-nginx-proxy-with-minio.html

dnf install zlib zlib-devel pcre pcre-devel openssl openssl-devel geoip geoip-devel
tar -xvzf nginx-1.24.0.tar.gz
cd nginx-1.24.0
./configure --prefix=/usr/local/nginx-1.24.0 --with-stream --with-threads --with-file-aio --with-http_ssl_module --with-http_realip_module --with-http_sub_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_auth_request_module
make
make install
ln -s /usr/local/nginx-1.24.0 /usr/local/nginx

修改nginx.conf配置文件中新增include vhost/*.conf;内容。

minio对象存储搭建与使用_nginx_04

在/usr/local/nginx/conf/新建vhost配置文件夹。

minio对象存储搭建与使用_html_05

新建minio.imddy.top.conf和minioconsole.imddy.top.conf文件

minio.imddy.top.conf

upstream minio_s3 {
        server 172.19.14.152:9000;
}

server {
        listen 80;	
        server_name minio.imddy.top;
        charset utf-8;
	    
        #ssl_session_timeout 5m;
        #ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        #ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:HIGH:!aNULL:!MD5:!RC4:!DHE;
        #ssl_prefer_server_ciphers on;
		
	#设定本虚拟主机的访问日志
        access_log  logs/minio.imddy.top.log ;

        error_page  404  /404.html;

        # redirect server error pages to the static page /50x.html
        #
	#定义错误提示页面
        error_page   500 502 503 504  /50x.html;
        	
        location / {
                proxy_set_header Host $host;
                proxy_set_header  X-Real-IP        $remote_addr;
                proxy_set_header REMOTE-HOST $remote_addr;
                proxy_set_header  X-Forwarded-For  $proxy_add_x_forwarded_for;
                proxy_pass http://minio_s3;
                proxy_redirect off;
                client_max_body_size    1000m;

                #client_body_timeout 600;
                #client_header_timeout 600;
                send_timeout 600;
                keepalive_timeout 600;
                proxy_connect_timeout 600;
                proxy_read_timeout 600;
                proxy_send_timeout 600;
        }
}

minioconsole.imddy.top.conf

upstream minio_console {
        server 172.19.14.152:9001;
}

server {
        listen 80;	
        server_name minioconsole.imddy.top;
        charset utf-8;
	    
        #ssl_session_timeout 5m;
        #ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        #ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:HIGH:!aNULL:!MD5:!RC4:!DHE;
        #ssl_prefer_server_ciphers on;
		
	#设定本虚拟主机的访问日志
        access_log  logs/minioconsole.imddy.top.log ;

        error_page  404  /404.html;

        # redirect server error pages to the static page /50x.html
        #
	#定义错误提示页面
        error_page   500 502 503 504  /50x.html;
        	
        location / {
                proxy_set_header Host $host;
                proxy_set_header  X-Real-IP        $remote_addr;
                proxy_set_header REMOTE-HOST $remote_addr;
                proxy_set_header  X-Forwarded-For  $proxy_add_x_forwarded_for;
                proxy_pass http://minio_console;
                proxy_redirect off;
                client_max_body_size    1000m;

                #client_body_timeout 600;
                #client_header_timeout 600;
                send_timeout 600;
                keepalive_timeout 600;
                proxy_connect_timeout 600;
                proxy_read_timeout 600;
                proxy_send_timeout 600;
        }
}

使用官方的推荐的代理配置

minio.imddy.top.conf官方推荐代理配置

upstream minio_s3 {
        server 172.19.14.152:9000;
}

server {
        listen 80;	
        server_name minio.imddy.top;
        charset utf-8;
        
        # Allow special characters in headers
        ignore_invalid_headers off;
        # Allow any size file to be uploaded.
        # Set to a value such as 1000m; to restrict file size to a specific value
        client_max_body_size 0;
        # Disable buffering
        proxy_buffering off;
        proxy_request_buffering off;
        
        location / {
                proxy_set_header Host $http_host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto $scheme;
                
                proxy_connect_timeout 300;
                # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
                proxy_http_version 1.1;
                proxy_set_header Connection "";
                chunked_transfer_encoding off;
                
                proxy_pass http://minio_s3; # This uses the upstream directive definition to load balance
        }
	    
}

minioconsole.imddy.top.conf官方推荐代理配置

upstream minio_console {
        server 172.19.14.152:9001;
}

server {
        listen 80;	
        server_name minioconsole.imddy.top;
        charset utf-8;
	
        # Allow special characters in headers
        ignore_invalid_headers off;
        # Allow any size file to be uploaded.
        # Set to a value such as 1000m; to restrict file size to a specific value
        client_max_body_size 0;
        # Disable buffering
        proxy_buffering off;
        proxy_request_buffering off;
        
        location / {
                proxy_set_header Host $http_host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto $scheme;
                proxy_set_header X-NginX-Proxy true;

                # This is necessary to pass the correct IP to be hashed
                real_ip_header X-Real-IP;
                
                proxy_connect_timeout 300;
                
                # To support websocket
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection "upgrade";
                
                chunked_transfer_encoding off;
                
                proxy_pass http://minio_console/; # This uses the upstream directive definition to load balance
        }    

}

minio对象存储搭建与使用_nginx_06

上传test001.jpg

minio对象存储搭建与使用_linux_07

设置bucket001桶为public

minio对象存储搭建与使用_linux_08

访问http://minio.imddy.top/bucket001/test001.jpg

minio对象存储搭建与使用_html_09


三、使用java的minio-sdk来实现文件上传和下载

https://min.io/docs/minio/linux/developers/java/minio-java.html

<dependency>
    <groupId>io.minio</groupId>
    <artifactId>minio</artifactId>
    <version>8.5.7</version>
</dependency>
import io.minio.BucketExistsArgs;
import io.minio.MakeBucketArgs;
import io.minio.MinioClient;
import io.minio.UploadObjectArgs;
import io.minio.errors.MinioException;
import java.io.IOException;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;

public class FileUploader {
  public static void main(String[] args)
      throws IOException, NoSuchAlgorithmException, InvalidKeyException {
    try {
      // Create a minioClient with the MinIO server playground, its access key and secret key.
      MinioClient minioClient =
          MinioClient.builder()
              .endpoint("https://play.min.io")
              .credentials("Q3AM3UQ867SPQQA43P2F", "zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG")
              .build();

      // Make 'asiatrip' bucket if not exist.
      boolean found =
          minioClient.bucketExists(BucketExistsArgs.builder().bucket("asiatrip").build());
      if (!found) {
        // Make a new bucket called 'asiatrip'.
        minioClient.makeBucket(MakeBucketArgs.builder().bucket("asiatrip").build());
      } else {
        System.out.println("Bucket 'asiatrip' already exists.");
      }

      // Upload '/home/user/Photos/asiaphotos.zip' as object name 'asiaphotos-2015.zip' to bucket
      // 'asiatrip'.
      minioClient.uploadObject(
          UploadObjectArgs.builder()
              .bucket("asiatrip")
              .object("asiaphotos-2015.zip")
              .filename("/home/user/Photos/asiaphotos.zip")
              .build());
      System.out.println(
          "'/home/user/Photos/asiaphotos.zip' is successfully uploaded as "
              + "object 'asiaphotos-2015.zip' to bucket 'asiatrip'.");
    } catch (MinioException e) {
      System.out.println("Error occurred: " + e);
      System.out.println("HTTP trace: " + e.httpTrace());
    }
  }
}

https://min.io/docs/minio/linux/developers/java/API.html

Create MinIO Client.

MinIO

MinioClient minioClient =
    MinioClient.builder()
        .endpoint("https://play.min.io")
        .credentials("Q3AM3UQ867SPQQA43P2F", "zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG")
        .build();

AWS S3

MinioClient minioClient =
    MinioClient.builder()
        .endpoint("https://s3.amazonaws.com")
        .credentials("YOUR-ACCESSKEYID", "YOUR-SECRETACCESSKEY")
        .build();

对于bucket常见的有创建bucket,判断bucket,list显示bucket等;

对于object常见的有,getObject,putObject,uploadObject,downloadObject,listObjects,removeObject,removeObjects等;

uploadObject,和downloadObject是一对,直接到文件;putObject和getObject是一对,这个操作的是流;

案例

application.properties

#minio.endpoint=https://play.min.io
minio.endpoint=http://minio.imddy.top
minio.accessKey=Q3AM3UQ867SPQQA43P2F
minio.secretKey=zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG
minio.bucketName=bucket001

MinioConfig.java

import io.minio.MinioClient;
import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.stereotype.Component;

@Data
@Configuration
@Component
@ConfigurationProperties(prefix = "minio")
public class MinioConfig {

    private String endpoint;
    private String accessKey;
    private String secretKey;
    private String bucketName;

    @Bean
    public MinioClient minioClient() {
        return MinioClient.builder()
                .endpoint(endpoint)
                .credentials(accessKey,secretKey)
                .build();
    }

}

MinioService.java

package com.imddysc.test.service;

import com.imddysc.test.config.MinioConfig;
import io.minio.BucketExistsArgs;
import io.minio.GetObjectArgs;
import io.minio.GetObjectResponse;
import io.minio.MakeBucketArgs;
import io.minio.MinioClient;
import io.minio.PutObjectArgs;
import io.minio.UploadObjectArgs;
import io.minio.errors.ErrorResponseException;
import io.minio.errors.InsufficientDataException;
import io.minio.errors.InternalException;
import io.minio.errors.InvalidResponseException;
import io.minio.errors.ServerException;
import io.minio.errors.XmlParserException;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.web.multipart.MultipartFile;

import java.io.IOException;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.util.UUID;

@Service
@Slf4j
public class MinioService {

    @Autowired
    private MinioClient minioClient;
    @Autowired
    private MinioConfig minioConfig;

    public void test001() {
        log.info("MinioService test001... ");
    }

    public void createBucket(String bucketName) throws Exception{
        if (!minioClient.bucketExists(BucketExistsArgs.builder().bucket(bucketName).build())) {
            minioClient.makeBucket(MakeBucketArgs.builder().bucket(bucketName).build());
        }
    }
    public String uploadFile(String bucketName, MultipartFile file) throws Exception {
        //String filename = UUID.randomUUID().toString()+"-"+file.getOriginalFilename().substring(file.getOriginalFilename().lastIndexOf("/"));
        int s1 = file.getOriginalFilename().lastIndexOf("/");
        int s2 = file.getOriginalFilename().lastIndexOf("\\");
        int sMax = Math.max(s1,s2);
        if (sMax != 0) {
            sMax = sMax+1;
        }
        String filename = UUID.randomUUID().toString()+"-"+file.getOriginalFilename().substring(sMax);
        minioClient.putObject(PutObjectArgs.builder()
                .bucket(bucketName)
                .object(filename)
                .stream(file.getInputStream(), file.getInputStream().available(), -1)
                .contentType(file.getContentType())
                .build());
        return minioConfig.getEndpoint() + "/" + bucketName + "/" + filename;
    }

    public GetObjectResponse downloadFile(String bucketName, String filename) throws Exception {
        return minioClient.getObject(GetObjectArgs.builder()
                .bucket(bucketName)
                .object(filename)
                .build());
    }


    public String uploadFile(String bucketName, String file) throws IOException, ServerException, InsufficientDataException, ErrorResponseException, NoSuchAlgorithmException, InvalidKeyException, InvalidResponseException, XmlParserException, InternalException {
        //String filename = UUID.randomUUID().toString()+file.substring(file.lastIndexOf("."));
        int s1 = file.lastIndexOf("/");
        int s2 = file.lastIndexOf("\\");
        int sMax = Math.max(s1,s2);
        if (sMax != 0) {
            sMax = sMax+1;
        }
        String filename = UUID.randomUUID().toString()+"-"+file.substring(sMax);
        log.info("filename: " + filename);
        minioClient.uploadObject(UploadObjectArgs.builder()
                .bucket(bucketName)
                .object(filename)
                .filename(file).build());
        return minioConfig.getEndpoint() + "/" + bucketName + "/" + filename;
    }

}


package com.imddysc.test;

import com.imddysc.test.service.MinioService;
import io.minio.GetObjectResponse;
import io.minio.errors.ErrorResponseException;
import io.minio.errors.InsufficientDataException;
import io.minio.errors.InternalException;
import io.minio.errors.InvalidResponseException;
import io.minio.errors.ServerException;
import io.minio.errors.XmlParserException;
import org.junit.jupiter.api.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.core.StringRedisTemplate;

import java.io.IOException;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;

@SpringBootTest
public class ApplicationTest {
    private static final Logger logger = LoggerFactory.getLogger(ApplicationTest.class);
    @Autowired
    private MinioService minioService;

    @Test
    public void test009() throws Exception {
        String s = minioService.uploadFile("bucket001", "C:\\Users\\Administrator\\Pictures\\微信图片_20230624012557.jpg");
        System.out.println("s: " + s);
    }
}

minio对象存储搭建与使用_nginx_10

http://minio.imddy.top/bucket001/61c135ee-09bc-4c05-81c3-b2100f82df0a-%E5%BE%AE%E4%BF%A1%E5%9B%BE%E7%89%87_20230624012557.jpg

minio对象存储搭建与使用_html_11




【版权声明】本文内容来自摩杜云社区用户原创、第三方投稿、转载,内容版权归原作者所有。本网站的目的在于传递更多信息,不拥有版权,亦不承担相应法律责任。如果您发现本社区中有涉嫌抄袭的内容,欢迎发送邮件进行举报,并提供相关证据,一经查实,本社区将立刻删除涉嫌侵权内容,举报邮箱: cloudbbs@moduyun.com

  1. 分享:
最后一次编辑于 2023年12月09日 0

暂无评论

推荐阅读
  6YY0QMPUXEwu   2023年12月10日   28   0   0 linux网卡
  jnZtF7Co41Wg   2023年12月11日   27   0   0 nginx客户端服务端
  stLBpDewCLT1   2023年12月08日   27   0   0 nginx
  nIt0XG0acU8j   2023年12月11日   29   0   0 linuxhtop
  jnZtF7Co41Wg   2023年12月10日   20   0   0 nginx客户端服务端NFS
  nIt0XG0acU8j   2023年12月09日   33   0   0 linuxsort
oIa1edJoFmXP