对于安卓系统的执法记录仪来说, 从设备上拷贝或上传录像文件是必不可少的功能，实时音视频在网络不好的情况可能会丢包或丢帧，导致实时回传的音视频不完整, 实际场景中实时视频回传的同时本地也录一份, 设备上有录像文件，录像检索，录像文件上传等这些配套功能都要有.
GB28181标准定义了历史视音频文件下载的详细信令流程，对于安卓设备端, 只要关注和媒体流发送者相关的流程就好:
1:SIP服务器向安卓设备发送Invite请求,请求中携带SDP信息,SDP中的s字段为“Download”代表文件下载,u字段代表下载通道ID和下载类型,t字段代表下载时间段,增加y字段描述SSRC值,f字段描述媒体参数,可扩展a字段携带下载倍速,将倍速参数传递给设备,若不携带倍速参数，默认为1倍速。
2:安卓设备收到Invite请求后,回复200 OK响应(安卓设备在回复200 OK等最终响应前，也可先回复一个临时响应，比如180 Ringing等), 携带SDP消息体,消息体中描述了媒体流发送者发送媒体流的IP、端口、媒体格式、SSRC字段等内容,可扩展a字段携带文件大小参数。
3:SIP服务器收到安卓设备返回的200 OK响应后,向安卓设备发送ACK请求,请求中不携带消息体,完成与安卓设备的Invite会话建立过程。
4:安卓设备按Invite SDP中给出的下载倍速发送音视频RTP包(推荐PS RTP包)到媒体服务器, 在RTP包发送过程中,下载端也可以通过SIP-INFO方法和MANSRTSP协议来更改下载速度.(需要注意的是:SDP属性a=downloadspeed取值集合是:{1, 2, 4}, MANSRTSP Scale头域取值集合是:{0.25, 0.5, 1, 2, 4}, 代码实现也要把{0.25, 0.5}倍速考虑进去)。
5:安卓设备发送完所有RTP包后, 需要发送一条会话内Message消息,通知SIP服务器下载已结束(对于会话内消息，具体请参考RFC3261-12.2 Requests within a Dialog)。
6:SIP服务器向安卓设备发送BYE消息。
7:安卓设备收到BYE消息后回复200 OK响应,会话断开。流程不算复杂，重点是代码实现.
GB28181标准、SIP和SDP等一大堆文档读起来枯燥乏味又费时间, 直接看例子上手快，下面给出具体实例:
安卓设备收到的INVITE SDP:
v=0
o=64010000041310000137 0 0 IN IP4 192.168.0.103
s=Download
u=64010000041310000137:0
c=IN IP4 192.168.0.103
t=1692953530 1692953581
m=video 20006 TCP/RTP/AVP 96
a=rtpmap:96 PS/90000
a=recvonly
a=connection:new
a=setup:passive
a=downloadspeed:2
y=1900000007s=Download代表文件下载, 下载速度是2倍速，SSRC是:1900000007(SSRC第1位为历史或实时媒体流的标识位,0为实时,1为历史).
安卓设备回复200 OK携带的SDP:
v=0
o=64010000041310000137 0 0 IN IP4 192.168.0.109
s=AndroidTest
c=IN IP4 192.168.0.109
t=0 0
m=video 15002 TCP/RTP/AVP 96
a=rtpmap:96 PS/90000
a=filesize:362489171
a=setup:active
a=connection:new
a=sendonly
y=1900000007a=filesize:362489171表示录像文件大小是362489171字节(Byte), 携带文件大小参数, 便于媒体流接收者计算下载进度(a=filesize是整个媒体容器的大小和实际发送的音视频帧总字节数有一定差异)。
文件下载过程中通过SIP-INFO消息和MANSRTSP协议将2倍速降低到0.25倍速:
PLAY RTSP/1.0
CSeq: 31129
Scale: 0.25安卓设备发送完所有音视频帧后发送会话内Message消息:
<?xml version="1.0" encoding="GB2312"?>
<Notify>
<CmdType>MediaStatus</CmdType>
<SN>793278085</SN>
<DeviceID>64010000041310000137</DeviceID>
<NotifyType>121</NotifyType>
</Notify>通知事件类型是"121", 表示历史媒体文件发送结束。
相关实现代码:
/*
* Copyright (C) 1130758427@qq.com. All rights reserved.
*/
/**
* 部分信令接口
*/
package com.gb.ntsignalling;
public interface GBSIPAgent {
void addDownloadListener(GBSIPAgentDownloadListener downloadListener);
void removeDownloadListener(GBSIPAgentDownloadListener removeListener);
/*
*响应Invite Download 200 OK
*/
boolean respondDownloadInviteOK(long id, String deviceId, String startTime, String stopTime, MediaSessionDescription localMediaDescription);
/*
*响应Invite Download 其他状态码
*/
boolean respondDownloadInvite(int statusCode, long id, String deviceId, String startTime, String stopTime);
/*
* 媒体流发送者在文件下载结束后发Message消息通知SIP服务器回文件已发送完成
* notifyType 必须是"121“
*/
boolean notifyDownloadMediaStatus(long id, String deviceId, String startTime, String stopTime, String notifyType);
/*
*终止Download会话
*/
void terminateDownload(long id, String deviceId, String startTime, String stopTime, boolean isSendBYE);
/*
*终止所有Download会话
*/
void terminateAllDownloads(boolean isSendBYE);
}
/**
* 信令Download Listener
*/
package com.gb.ntsignalling;
public interface GBSIPAgentDownloadListener {
/*
*收到s=Download的文件下载Invite
*/
void ntsOnInviteDownload(long id, String deviceId, SessionDescription sessionDescription);
/*
*发送Download invite response 异常
*/
void ntsOnDownloadInviteResponseException(long id, String deviceId, String startTime, String stopTime, int statusCode, String errorInfo);
/*
* 收到CANCEL Download INVITE请求
*/
void ntsOnCancelDownload(long id, String deviceId, String startTime, String stopTime);
/*
* 收到Ack
*/
void ntsOnAckDownload(long id, String deviceId, String startTime, String stopTime);
/*
* 更改下载速度
*/
void ntsOnDownloadMANSRTSPScaleCommand(long id, String deviceId, String startTime, String stopTime, double scale);
/*
* 收到Bye
*/
void ntsOnByeDownload(long id, String deviceId, String startTime, String stopTime);
/*
* 不是在收到BYE Message情况下， 终止Download
*/
void ntsOnTerminateDownload(long id, String deviceId, String startTime, String stopTime);
/*
* Download会话对应的对话终止, 一般不会触发这个回调,目前只有在响应了200K, 但在64*T1时间后还没收到ACK,才可能会出发
收到这个, 请做相关清理处理
*/
void ntsOnDownloadDialogTerminated(long id, String deviceId, String startTime, String stopTime);
}
/**
* 部分JNI接口, rtp ps 打包发送等代码C++实现
*/
package com.daniulive.smartpublisher;
public class SmartPublisherJniV2 {
/**
* Open publisher(启动推送实例)
*
* @param ctx: get by this.getApplicationContext()
*
* @param audio_opt:
* if 0: 不推送音频
* if 1: 推送编码前音频(PCM)
* if 2: 推送编码后音频(aac/pcma/pcmu/speex).
*
* @param video_opt:
* if 0: 不推送视频
* if 1: 推送编码前视频(NV12/I420/RGBA8888等格式)
* if 2: 推送编码后视频(AVC/HEVC)
* if 3: 层叠加模式
*
* <pre>This function must be called firstly.</pre>
*
* @return the handle of publisher instance
*/
public native long SmartPublisherOpen(Object ctx, int audio_opt, int video_opt, int width, int height);
/**
* 设置流类型
* @param type: 0:表示 live 流, 1:表示 on-demand 流, SDK默认为0(live流)
* 注意: 流类型设置当前仅对GB28181媒体流有效
* @return {0} if successful
*/
public native int SetStreamType(long handle, int type);
/**
* 投递视频 on demand包, 当前只用于GB28181推送, 注意ByteBuffer对象必须是DirectBuffer
*
* @param codec_id: 编码id, 当前支持H264和H265, 1:H264, 2:H265
*
* @param packet: 视频数据, 包格式请参考H264/H265 Annex B Byte stream format, 例如:
* 0x00000001 nal_unit 0x00000001 ...
* H264 IDR: 0x00000001 sps 0x00000001 pps 0x00000001 IDR_nal_unit .... 或 0x00000001 IDR_nal_unit ....
* H265 IDR: 0x00000001 vps 0x00000001 sps 0x00000001 pps 0x00000001 IDR_nal_unit .... 或 0x00000001 IDR_nal_unit ....
*
* @param offset: 偏移量
* @param size: packet size
* @param pts_us: 时间戳, 单位微秒
* @param is_pts_discontinuity: 是否时间戳间断，0:未间断，1:间断
* @param is_key: 是否是关键帧, 0:非关键帧, 1:关键帧
* @param codec_specific_data: 可选参数，可传null, 对于H264关键帧包, 如果packet不含sps和pps, 可传0x00000001 sps 0x00000001 pps
* ,对于H265关键帧包, 如果packet不含vps,sps和pps, 可传0x00000001 vps 0x00000001 sps 0x00000001 pps
* @param codec_specific_data_size: codec_specific_data size
* @param width: 图像宽, 可传0
* @param height: 图像高, 可传0
*
* @return {0} if successful
*/
public native int PostVideoOnDemandPacketByteBuffer(long handle, int codec_id,
ByteBuffer packet, int offset, int size, long pts_us, int is_pts_discontinuity, int is_key,
byte[] codec_specific_data, int codec_specific_data_size,
int width, int height);
/**
* 投递音频on demand包, 当前只用于GB28181推送, 注意ByteBuffer对象必须是DirectBuffer
*
* @param codec_id: 编码id, 当前支持PCMA和AAC, 65536:PCMA, 65538:AAC
* @param packet: 音频数据
* @param offset：packet偏移量
* @param size: packet size
* @param pts_us: 时间戳, 单位微秒
* @param is_pts_discontinuity: 是否时间戳间断，0:未间断，1:间断
* @param codec_specific_data: 如果是AAC的话，需要传 Audio Specific Configuration
* @param codec_specific_data_size: codec_specific_data size
* @param sample_rate: 采样率
* @param channels: 通道数
*
* @return {0} if successful
*/
public native int PostAudioOnDemandPacketByteBuffer(long handle, int codec_id,
ByteBuffer packet, int offset, int size, long pts_us, int is_pts_discontinuity,
byte[] codec_specific_data, int codec_specific_data_size,
int sample_rate, int channels);
/**
* 启动 GB28181 媒体流
*
* @return {0} if successful
*/
public native int StartGB28181MediaStream(long handle);
/**
* 停止 GB28181 媒体流
*
* @return {0} if successful
*/
public native int StopGB28181MediaStream(long handle);
/**
* 关闭推送实例，结束时必须调用close接口释放资源
*
* @return {0} if successful
*/
public native int SmartPublisherClose(long handle);
}
/**
* Listener部分实现代码
*/
package com.daniulive.smartpublisher;
public class RecordDownloadListenerImpl implements com.gb.ntsignalling.GBSIPAgentDownloadListener {
/*
*收到s=Download的文件下载Invite
*/
@Override
public void ntsOnInviteDownload(long id, String deviceId, SessionDescription sdp) {
if (!post_task(new OnInviteTask(this.context_, this.is_exit_, this.senders_map_, deviceId, sdp, id))) {
Log.e(TAG, "ntsOnInviteDownload post_task failed, " + RecordSender.make_print_tuple(id, deviceId, sdp.getTime().getStartTime(), sdp.getTime().getStopTime()));
// 这里不发488, 等待事务超时也可以的
GBSIPAgent agent = this.context_.get_agent();
if (agent != null)
agent.respondDownloadInvite(488, id, deviceId, sdp.getTime().getStartTime(), sdp.getTime().getStopTime());
}
}
/*
* 收到CANCEL Download INVITE请求
*/
@Override
public void ntsOnCancelDownload(long id, String deviceId, String startTime, String stopTime) {
Log.i(TAG, "ntsOnCancelDownload, " + RecordSender.make_print_tuple(id, deviceId, startTime, stopTime));
RecordSender sender = senders_map_.remove(id);
if (null == sender)
return;
StopDisposeTask task = new StopDisposeTask(sender);
if (!post_task(task))
task.run();
}
/*
* 收到Ack
*/
@Override
public void ntsOnAckDownload(long id, String deviceId, String startTime, String stopTime) {
Log.i(TAG, "ntsOnAckDownload, "+ RecordSender.make_print_tuple(id, deviceId, startTime, stopTime));
RecordSender sender = senders_map_.get(id);
if (null == sender) {
Log.e(TAG, "ntsOnAckDownload get sender is null, " + RecordSender.make_print_tuple(id, deviceId, startTime, stopTime));
GBSIPAgent agent = this.context_.get_agent();
if (agent != null)
agent.terminateDownload(id, deviceId, startTime, stopTime, false);
return;
}
StartTask task = new StartTask(sender, this.senders_map_);
if (!post_task(task))
task.run();
}
/*
* 收到Bye
*/
@Override
public void ntsOnByeDownload(long id, String deviceId, String startTime, String stopTime) {
Log.i(TAG, "ntsOnByeDownload, "+ RecordSender.make_print_tuple(id, deviceId, startTime, stopTime));
RecordSender sender = this.senders_map_.remove(id);
if (null == sender)
return;
StopDisposeTask task = new StopDisposeTask(sender);
if (!post_task(task))
task.run();
}
/*
* 更改下载速度
*/
@Override
public void ntsOnDownloadMANSRTSPScaleCommand(long id, String deviceId, String startTime, String stopTime, double scale) {
if (scale < 0.01) {
Log.e(TAG, "ntsOnDownloadMANSRTSPScaleCommand invalid scale:" + scale + " " + RecordSender.make_print_tuple(id, deviceId, startTime, stopTime));
return;
}
RecordSender sender = this.senders_map_.get(id);
if (null == sender) {
Log.e(TAG, "ntsOnDownloadMANSRTSPScaleCommand can not get sender, scale:" + scale + " " + RecordSender.make_print_tuple(id, deviceId, startTime, stopTime));
return;
}
sender.set_speed(scale);
Log.i(TAG, "ntsOnDownloadMANSRTSPScaleCommand, scale:" + scale + " " + RecordSender.make_print_tuple(id, deviceId, startTime, stopTime));
}
}
/**
* 发送文件部分代码
*/
package com.daniulive.smartpublisher;
public class RecordSender {
public void set_speed(double speed) {
int percent_speed = (int)(speed*100);
this.percent_speed_.set(percent_speed);
}
public void set_file_description(RecordFileDescription desc) {
this.file_description_ = desc;
}
public static String make_print_tuple(long id, String device_id, String start_time, String stop_time) {
StringBuilder sb = new StringBuilder(96);
sb.append("[id:").append(id);
sb.append(", device:" + device_id);
sb.append(", t=").append(start_time).append(" ").append(start_time);
sb.append("]");
return sb.toString();
}
public boolean start() {
SendThread current_thread = thread_.get();
if (current_thread != null) {
if (current_thread.is_exit()) {
Log.e(TAG, "start, the thread already exists and has exited, return false, " + get_print_tuple());
return false;
}
Log.i(TAG, "start, the thread already exists and has exited, return true, " + get_print_tuple());
return true;
}
SendThread thread = new SendThread();
if (!thread_.compareAndSet(null, thread)) {
Log.i(TAG, "start, call compareAndSet return false, the thread already exists, return true, " + get_print_tuple());
return true;
}
try {
Log.i(TAG, "start thread, " + get_print_tuple());
thread.start();
}catch (Exception e) {
thread_.compareAndSet(thread, null);
Log.e(TAG, "start e:", e);
return false;
}
return true;
}
public void stop() {
SendThread current_thread = thread_.get();
if (current_thread != null && !current_thread.is_exit()) {
current_thread.exit();
Log.i(TAG, "stop, exit thread " + get_print_tuple());
}
}
private boolean init_native_sender(StackDisposable disposables) {
if(native_handle_ !=0) {
Log.e(TAG, "init_native_sender, native_handle_ is not 0, " + get_print_tuple());
return false;
}
if (null == this.media_info_ || !this.media_info_.is_has_track() ) {
Log.e(TAG, "init_native_sender, there is no track, " + get_print_tuple());
return false;
}
if (0 == rtp_handle_) {
Log.e(TAG, "init_native_sender, rtp_handle_ is 0, " + get_print_tuple());
return false;
}
if (null == lib_publisher_){
Log.e(TAG, "init_native_sender, lib_publisher_ is null, " + get_print_tuple());
return false;
}
Context context = this.context_.get_context();
if (null == context) {
Log.e(TAG, "init_native_sender, context is null, " + get_print_tuple());
return false;
}
long handle = lib_publisher_.SmartPublisherOpen(context, media_info_.is_has_audio_track()?2:0, media_info_.is_has_video_track()?2:0, 0, 0);
if (0 == handle) {
Log.e(TAG, "init_native_sender, call SmartPublisherOpen failed, " + get_print_tuple());
return false;
}
NativeSenderDisposable native_disposable = new NativeSenderDisposable(lib_publisher_, handle);
lib_publisher_.SetStreamType(handle, 1);
List<MediaTrack> tracks = media_info_.get_tracks();
for (MediaTrack i : tracks) {
if (i.is_video())
lib_publisher_.SetEncodedVideoCodecId(handle, i.codec_id(), i.csd_set(), i.csd_set() != null? i.csd_set().length : 0);
else if(i.is_audio())
lib_publisher_.SetEncodedAudioCodecId(handle, i.codec_id(), i.csd_set(), i.csd_set() != null? i.csd_set().length : 0);
}
lib_publisher_.SetGB28181RTPSender(handle, rtp_handle_, rtp_payload_type_, rtp_encoding_name_);
int ret = lib_publisher_.StartGB28181MediaStream(handle);
if (ret != 0) {
Log.e(TAG, "init_native_sender, call StartGB28181MediaStream failed, " + get_print_tuple());
native_disposable.dispose();
return false;
}
native_disposable.is_need_call_stop(true);
disposables.push(native_disposable);
native_handle_ = handle;
return true;
}
private boolean post_media_packet(MediaPacket packet) {
/*Log.i(TAG, "post "+ MediaTrack.get_media_type_string(packet.media_type()) + " " +
MediaTrack.get_codec_id_string(packet.codec_id()) + " packet, pts:" + out_point_3(packet.pts_us()/1000.0) +"ms, key:"
+ (packet.is_key()?1:0) + ", size:" + packet.size()); */
if (null == lib_publisher_ || 0 == native_handle_ || !packet.is_has_data())
return false;
if (packet.is_audio()) {
if (packet.is_aac()) {
if (packet.is_has_codec_specific_data_set())
return 0 == lib_publisher_.PostAudioOnDemandPacketByteBuffer(native_handle_, packet.codec_id(), packet.data(), 0, packet.size(),
packet.pts_us(), 0, packet.codec_specific_data_set(), packet.codec_specific_data_set_size(), 0, 0);
}
}else if (packet.is_video()) {
if (packet.is_avc() || packet.is_hevc()) {
return 0 == lib_publisher_.PostVideoOnDemandPacketByteBuffer(native_handle_, packet.codec_id(), packet.data(),
0, packet.size(), packet.pts_us(), 0, packet.is_key()?1:0,
packet.codec_specific_data_set(), packet.codec_specific_data_set_size(), 0, 0);
}
}
return false;
}
private void release_packets(Deque<MediaPacket> packets) {
while (!packets.isEmpty())
packets.removeFirst().release_buffer();
}
private static String out_point_3(double v) { return String.format("%.3f", v); }
public static String to_mega_bytes_string(long bytes) {
double mb = bytes/(1024*1024.0);
return out_point_3(mb);
}
private class SendThread extends Thread {
@Override
public void run() {
/***
*相关代码
**/
}
}
}我的安卓GB28181视音频文件下载实现代码较多，大体上分为三块，信令部分，rtp发送打包部分，文件检索发送等逻辑, 有些用Java实现有些用C++, 这里为了方便说明信令流程和关键细节, 只给出部分接口定义和实现代码, 如有不清楚的地方请联系qq: 1130758427。
从流程描述到一个稳定可靠的代码实现, 要花不少时间和精力。文件下载和实时音视频上传的侧重点不一样，实时音视频播放主要考虑低延时，录像文件下载要尽可能保证音视频的完整性，我的下载实现是少丢帧少丢包，网络不好时优先考虑降速, 另外文件下载推荐使用TCP传输rtp包, 现在常用的点播协议如HLS、DASH等基本都用HTTP传，点播用RTP打包传的不多, 所以我强烈推荐使用RTP over TCP(具体请参考RFC 4571)。