Monthly Archives: June 2014

X264 encoding example: One pass

One pass x264 encoding example. X264 is best video encoding algorithm out there in all respect. It has very good presets for almost all of your needs. You can search for all the options in X264 wiki and try to understand them. There is a lot of individual configuration option but it is like setting up a good PC. If you are selecting everything correctly but one incorrect choice may lead to unexpected behaviour. So it is recommended to use presets rather than fiddling with individual parameters.

Anyhow you can check below structure in x264.h file and see what all you can get. Detailed description are present on wiki.

typedef struct x264_param_t
{
/* CPU flags */
unsigned int cpu;

..
Some devices are just capable of playing low preset so you have to look into that. The encoding as any video encoding is CPU intensive process. Just for sake of trying I tried to encode on quad-core android device with 1 GB RAM and it was like 1 frame per second or even less.

I have added my encoder file in config.mak or you can even generate makefile once and then add your new source files there and do make to compile everything seamlessly.

The way to use below encoding file is very simple. external_api has the easy interface. Just call set parameters first to initialize the codec with all parameters and then call initialize pic_in pic_out after that on each YUV frame just call encode. encode will return you the encoded x264 data for that particular frame. First frame data has the program table SPS so you have to send part of the first frame if you want to play from any in between data. This gives you just x264 elementary stream with no playing information. You can save this stream in .h264 file format and VLC can play it very fast.

I was using the dll generated on windows with C# code but you can just make few changes here and there or simply call the encoder file functions directly with your header file to use inside C or C++ project. I even modified this to use with JNI in android.

Filename: external_api.h

/*******************************************************
* Readme for x264 encoding
* call setX264Params first to set the encoder parameters
* call initializePicIn and initializePicOut to initialize pics
*/
#define WIN32_LEAN_AND_MEAN
#define _WIN32_WINNT 0x0501
#include 
#include 
#include 
#include 
#include 

#include           
#include          
#include          // Needed for exit()
#ifdef WIN
  #include       // Needed for all Winsock stuff
#endif
#ifdef BSD
  #include     // Needed for sockets stuff
  #include    // Needed for sockets stuff
  #include    // Needed for sockets stuff
  #include     // Needed for sockets stuff
  #include         // Needed for sockets stuff
  #include         // Needed for sockets stuff
#endif

#include "stdint.h"
#include "stdio.h"
#include "stdlib.h"
#include "x264.h"

//  Microsoft
#define DLLEXPORT __declspec(dllexport)
#define CDECL  __cdecl


DLLEXPORT x264_t* CDECL setX264Params(int width, int height, int FPS, int level);
DLLEXPORT uint8_t* CDECL encode(int w, int h, const uint8_t* data, x264_t* encoder, x264_picture_t* pic_in_p, x264_picture_t* pic_out_p, int frameNum, int* retVal);

DLLEXPORT void CDECL clean(x264_picture_t* pic_in_p);

DLLEXPORT x264_picture_t* CDECL initializePicOut();
DLLEXPORT x264_picture_t* CDECL initializePicIn(int WIDTH, int HEIGHT);

DLLEXPORT void CDECL send_streaming_data_tcp(char* out_buf, int size, void* ConnectSocket);
DLLEXPORT SOCKET* CDECL set_up_tcp_client( );

DLLEXPORT int CDECL close_connection_tcp(SOCKET ConnectSocket);

DLLEXPORT int CDECL write_packet_tcp(void* socket, uint8_t *buf, int buf_size);

DLLEXPORT struct user_data* CDECL set_up_udp_client();

DLLEXPORT int CDECL close_udp_connection(int client_s);
DLLEXPORT void CDECL send_streaming_data_udp(char* out_buf, int size, int client_s, struct sockaddr_in* server_addr);

File name encoder.c:


//compile:  just do make
//some changes in config.mak file
//IMPLIBNAME=libx264.dll.a
//SOFLAGS= -lWs2_32 -liconv -shared -Wl,--out-implib,$(IMPLIBNAME) -Wl,--enable-auto-image-base 
//default: lib-shared
//install: install-lib-shared
//LDFLAGSCLI = -lshell32 -lWs2_32
//CLI_LIBX264 = $(LIBX264)

//#define X264_API_IMPORTS
#define NUM_CPU 4
#define DEF_PRESET "superfast"
#define DEF_TUNE "zerolatency"
//other options //veryfast //crashing in ultrafast //best at superfast//slow in veryslow
                //zerolatency //crashes with fastdecode
#define I_FPS_DEN 1
#define B_INTRA_REFRESH 1
#define I_SCENECUT_THRESHOLD 40
#define B_REPEAT_HEADERS 1
#define B_ANNEXB 1

#include "external_api.h"

DLLEXPORT x264_t* CDECL setX264Params(int width, int height, int FPS, int level)
{
  //level will be used for different encoding setting
  //experiment more to fix three level of setting
  x264_param_t param;
  int res = 0;
  res = x264_param_default_preset(&param, DEF_PRESET, DEF_TUNE);//veryfast //crashing in ultrafast //best at superfast//slow in veryslow
                                                                //zerolatency //crashes with fastdecode
  if(res != 0) {
    printf("error: cannot set the default pre-set on x264.\n");
    return -1;
  }
  res = x264_param_apply_profile(&param, "baseline");
  if(res != 0) {
    printf("error: cannot set the baseline profile on x264.\n");
    return -2;
  }
  param.i_threads = 1.5 * NUM_CPU;// 1.5 * logical processors
  param.i_width = width;
  param.i_height = height;
  param.i_fps_num = FPS;
  param.i_fps_den = I_FPS_DEN;
  // Intra refres:
  param.i_keyint_max = FPS;
  param.b_intra_refresh = I_SCENECUT_THRESHOLD;//commenting makes it very fast
  param.i_scenecut_threshold = 40;
  //Rate control:
  param.rc.i_rc_method = X264_RC_CRF;
  param.rc.f_rf_constant = FPS-5;
  param.rc.f_rf_constant_max = FPS + 5;
  //For streaming:
  param.b_repeat_headers = B_REPEAT_HEADERS;
  param.b_annexb = B_ANNEXB;

  // initialize the encoder
  x264_t* encoder = x264_encoder_open(&param);
  if(!encoder) 
  {
    printf("error: cannot create the encoder.\n");
    return -3;
  }

  return encoder;
}

DLLEXPORT x264_picture_t* CDECL initializePicIn(int WIDTH, int HEIGHT)
{
  x264_picture_t* pic_in_p = malloc(sizeof(x264_picture_t));

  x264_picture_alloc(pic_in_p, X264_CSP_I420, WIDTH, HEIGHT);
  x264_picture_init(pic_in_p);
  pic_in_p->img.i_csp = X264_CSP_I420;

  pic_in_p->img.i_stride[0] = WIDTH*HEIGHT;
  pic_in_p->img.i_stride[1] = WIDTH*HEIGHT;
  pic_in_p->img.i_stride[2] = WIDTH*HEIGHT;
  pic_in_p->img.i_plane = 3;
  return pic_in_p;
}

DLLEXPORT x264_picture_t* CDECL initializePicOut()
{
    x264_picture_t* pic_out_p = malloc(sizeof(x264_picture_t));
	return pic_out_p;
}

//TODO: clean pic_out also
DLLEXPORT void CDECL clean(x264_picture_t* pic_in_p)
{
    x264_picture_clean(pic_in_p);
}

//TODO: check for frameNum //check whether it works without framenumber 

DLLEXPORT uint8_t* CDECL encode(int w, int h, const uint8_t* data, x264_t* encoder, x264_picture_t* pic_in_p, x264_picture_t* pic_out_p, int frameNum, int* retVal)
{
  int y_bytes = w * h;
  int uv_bytes = w * h / 4;
 
  pic_in_p->img.i_csp = X264_CSP_I420;
  pic_in_p->img.i_stride[0] = w;
  pic_in_p->img.i_stride[1] = w >> 1;
  pic_in_p->img.i_stride[2] = w >> 1;
  pic_in_p->img.i_plane = 3;
  pic_in_p->img.plane[0] = data;
  pic_in_p->img.plane[1] = &(data[y_bytes]);
  pic_in_p->img.plane[2] = &(data[y_bytes + uv_bytes]);

  pic_in_p->i_pts = frameNum;

  x264_nal_t* nals = NULL;
  int i_nals;
  *retVal = x264_encoder_encode(encoder, &nals, &i_nals, pic_in_p, pic_out_p);

  return nals[0].p_payload;
}

Source Insight: A great code browsing tool

The best took for code analysis. Irrespective of code size it gives you such simplicity in browsing the code as it seems like everything is just written in ten – twenty A4 sheets in big letters and a few page flips is all you need to find anything. There is always a learning curve for each took but once you get the small short cuts. It become so easy to browse the code. Can I think of any more short cut or stuff from a code browser? no I guess not. Best tool in market for code browsing so go for it. Very handy and fast. Find all references
ctrl + /
Highlight:
shift + F8 . . . ,, ;;;

Wireshark: a great handy tool always

Wireshark is a free tool for your networking need. In Any kind of network related development wireshark comes handy in a great way. You can use it for debugging a network related project. You can go deep and see why your application throughput is not as expected. What is the reason for retransmission how is network behaving overall.

There are multiple GUI features to help you analyse the network traffic. Some example is stevens graph, RTT graph etc. These graphs gives instant insight into network and your application behaviour.
You can check for window size for TCP congestion period nack, fast retransmission. You can get your specific packets by applying easy filters based on protocol, port number etc.

It has good color code to distinguish the types of packets. Althogh sometime the analysis done by wireshark on packets are not correct like may be the packet which wireshark think as ack is actually a data packet. This is just because the way wireshark is interpreting the packet may be based on size field or some history of the network.