Summary:

1. Motivation
2. Measuring procedure
3. Results
4. Future developments
5. Implementation
6. Code

1. Motivation

One of the essential features of TCP is the sliding window algorithm. The congestion window size defines the amount of data, in segments, that a TCP sender can transmit before it has to wait for an acknowledgment to proceed. It provides the Transport level with a flow control mechanism (end-to-end) and it makes sure that data is correct and delivered in the right order.

During a TCP connection, the window size is adjusted according to the receiver's ability to handle incoming TCP segments, which essentially depends on its memory resources (and CPU speed). It starts with a congestion window size of 1 segment that grows with time if everything works well.

This mechanism works over one TCP connection, but the size of the window starts again with its initial 1 segment value for every new connection. We know that an HTTP session is based on opening and closing numerous very short TCP connections between a client and a server. Given that, an HTTP session cannot take advantage of the sliding window mechanism, since it will rarely go further than a few round-trips, which does not allow the window size to grow significantly, even though resources might be available to increase efficiency.

We want to keep track of the TCP congestion window size over several connections between two computers, and over a given period of time. This way, the regular mechanism could go further than only a few steps and larger window sizes could be used, decreasing overhead and therefore improving efficiency.
 

2. Measuring procedure

To check the validity of the above idea, we needed to set up a way of measuring and visualizing the gain we would achieve over a traditional implementation of TCP.

The mechanism remembering the congestion window size over several TCP connections was implemented on a machine that would simulate an HTTP server.

Another machine had to simulate an HTTP client. Since local measurements, e.g. using computers hooked up to the same LAN, proved not to be appropriate because the round-trip time was so short that the gain was very small, we installed the client part in Paris, France, with the server machine located in Boston, MA. But this did not allow to see what would happen for intermediate distances. In order to simulate a variety of distances, a local client machine was modified so that it introduced a fixed delay every time a TCP connection acknowledged a packet. Thus we measured the behavior of the new mechanism both on one long distance connection and on a local connection simulating wider ones with delays.

Application tools were developed in order to simulate an HTTP server on the server machine and an HTTP client on the "delayed" machine: respectively httpsims and httpsimc.

3. Results
 
This procedure led to the following four graphs. Figure 1 compares the behavior of TCP with our mechanism with the behavior of the conventional TCP and clearly shows that we benefit from remembering the congestion windows size over several TCP connections.

However, this was done using only local measurements. Figure 2 shows the same thing on a "real" long distance connection (Boston/Paris).

To give a more accurate idea of the speedup in relation to the distance, Figure 3 shows the speedup related to the delays we introduced in order to simulate various distances. It shows it for four different amounts of data transmitted, ranging from 10 MB to 40 MB. We see that we get the best speedup for the biggest delay – the widest simulated network – and that the gain is lower for bigger amounts of transmitted data. Our mechanism's performance is at its best when the round-trip time gets longer.

Figure 4 demonstrates better what actually happens to the congestion window size. It shows the evolution of the congestion window size over time with and without the remembering mechanism: the regular TCP behavior is cyclic over several connections whereas the new implementation keeps increasing the congestion window size.

4. Future developments

A few ideas:

• The way stored congestion window sizes are eventually discarded (see section 5.1 below) is quite raw and could be refined. We could use a decay function based on some "forgetfulness factor" computed with data collected during previous connections in order to decide when to stop using a window size.
  
  
• Congestion window sizes are collected only every time a TCP connection ends. Collecting them throughout a connection would be more robust and might be required to compute the above decay function.

5. Implementation

5.1 Implementation of the Congestion Window Size Remembering mechanism

This was done on the server machine.

In Linux, the congestion window size is stored in the variable cong_window in the sock structure (in linux/include/net/sock.h).

A linked list was implemented in kernel memory with a set of functions to store and search congestion window sizes. Three pieces of information are saved:

• the IP address of the peer computer;
• the size of the congestion window;
• a Linux time stamp (seconds since 1/1/1970).

• LIFETIME_WIND: for how many seconds should we keep window sizes in memory?
• MAX_WINDLIST: how many of them do we want to store?

When the system boots, it behaves just like any Linux system. The capture of window sizes can be started and stopped manually using this trick: various operations implemented in the kernel will be performed whenever TCP sees some specific IP address. Therefore we can telnet to these addresses in order to get these operations completed:

• telnet cs1 starts the capture;
• telnet cs2 stops the capture (and frees the linked list).

Congestion window sizes must be used instead of the basic initial size (1 packet) for every incoming TCP connection. This is done in tcp_conn_request() (in linux/net/ipv4/tcp_input.c).

(See code in section 6.1)
 

5.2 Implementation of an extra feature that displays captured window sizes

Getting data out of kernel space is not an easy job. Conveniently, the Proc file system provides a way of accessing kernel data from user space. Kernel variables such as max_files are already available by accessing various pseudo-files in /proc (for instance /proc/sys/kernel/file-max for max_files).

Creating a new entry – a new pseudo-file in /proc – is simple: we just need to update a few structures and functions in linux/include/linux/proc_fs.h, linux/fs/proc/root.c and linux/fs/proc/array.c and write a function that will fill in a page with what we want to display whenever invoked.

The entry added in /proc for this project is called project and the function it calls is get_project_status() (implemented in linux/net/ipv4/tcp.c).

To display the linked list (IP address, window size, age), we type:

• cat /proc/project

5.3 Implementation of delays on the client side

There already was a function in the Linux kernel that sent delayed ACKs under certain circumstances: tcp_send_delayed_ack().

Using the function in question to impose a fixed delay for every ACK was done by using this fixed delay whenever the function was already used (in tcp_queue() in linux/net/ipv4/tcp_input.c). We also had to call that function with our fixed delay in lieu of the regular tcp_send_ack(). The latter was done by renaming tcp_send_ack() into tcp_real_send_ack() and writing a new tcp_send_ack() that just called tcp_send_delayed_ack() (in linux/net/ipv4/tcp_output.c). Then, the tcp_real_send_ack() function had to be called when the delay expired (in linux/net/ipv4/tcp_timer.c).

Note that the acknowledgment delay is defined as follows:

#define ACK_DELAY HZ/50 /* This amounts to a delay of 20 ms */

with HZ = 100 jiffies and that ACK_DELAY is later used as an integer. Thus, we can only introduce delays such as 10 ms, 20 ms, 30 ms, 40 ms, etc.

(See code in section 6.3)
 

5.4 Implementation of the measurement tools: httpsims, httpsimc and conv

The way httpsims and httpsimc work has been described in section 2. They have been implemented in a classic UNIX Client/Server fashion with httpsims forking off child processes to deal with every incoming connection.

(See code in sections 6.4.1 and 6.4.2)

From the data that tcpdump outputs, we want to extract the time when each TCP packet is sent from the server to the client and the amount of data that have been sent so far. To filter the tcpdump data, we use grep, cut and conv as follows:

tcpdump -l -tt -S src port 4200 and dst hereford | grep -v S | cut -d' ' -f1,6 | cut -c7- | cut -d: -f1 | grep -v ack | conv > file_to_plot

where hereford is the client machine address, 4200 is the port used by httpsims on the server machine and file_to_plot is the file we are going to plot (for example with gnuplot).

Without conv, the above sequence of commands would output a Linux time stamp and a TCP sequence number for each TCP packet. conv changes the time stamps so that they start from time 0 and it also changes the TCP sequence numbers into kilobytes, which also start from 0. Since the couple httpsims/httpsimc simulates HTTP by rapidly establishing and closing short TCP connections (sending a few kilobytes for each connection), conv also has to deal with the changes of TCP sequence numbers for each new connection, while adding up kilobytes of transmitted data.

(See code in section 6.4.3)

6. Code

6.1 Code for the Congestion Window Size Remembering mechanism

6.1.1 New definitions, declarations and prototypes

In linux/include/linux/tcp.h:

#define MAX_WINDLIST         4096 /* Number of cong_windows remembered */
#define LIFETIME_WIND        300  /* For how long to remember them (s) */
#define HEAD_LIST_ADDR       42   /* Name for the head of the list     */

typedef struct s_wind             /* Window storage structure          */
{
    __u32 addr;                   /* IP address  (sk->daddr)           */
    unsigned short window;        /* Window size (sk->cong_window)     */
    unsigned int date;            /* Linux time stamp (xtime.tv_sec)   */
    struct s_wind *next;
} WIND, *PTWIND;

extern PTWIND NewWindList(void);
extern PTWIND SeekWind(__u32 addr);
extern void   AddWind(__u32 addr, unsigned short window);
extern void   KillWindList(void);
extern void   GarbCollWindList(void);
extern void   CollectWind(struct sock *sk);
extern void   WindManager(struct sock *sk);

In linux/net/ipv4/tcp.c:

#include <asm/param.h>
#include <linux/malloc.h>
#include <linux/string.h>

PTWIND head_windlist = NULL;      /* Window linked list head */
PTWIND tail_windlist = NULL;      /* Window linked list tail */
int counter_windlist = 0;         /* Number of entries in Window list */
extern struct timeval xtime;      /* Added this instead of using sys_time() */

6.1.2 New functions

In linux/net/ipv4/tcp.c:

/* 
 *  NewWindList() function 
 *  Creates the head for the window sizes' linked list. 
 */ 
  
PTWIND NewWindList(void) 
{ 
    PTWIND tmp = NULL; 

    tmp = (PTWIND) kmalloc(sizeof(WIND), GFP_KERNEL); 
    if (tmp) 
    { 
        tmp->addr   = HEAD_LIST_ADDR; 
        tmp->window = 0; 
        tmp->date   = 0; 
        tmp->next   = NULL; 
    } 
    else 
        printk("NewWindList(): Out of memory."); 
        /* System crash? */ 

    return(tmp); 
} 

/* 
 *  SeekWind() function 
 *  Finds the last window size for a given address if we have it and 
 *  deletes outdated values. 
 * 
 *  Input: 
 *   - addr: (__u32) the address we're looking for 
 *  Output: 
 *   - a pointer to the cell embedding the size or NULL if not found 
 */ 

PTWIND SeekWind(__u32 addr) 
{ 
    PTWIND wind, prev; 
  
    wind = prev = head_windlist; 
  
    while (wind) 
    { 
        if (wind->addr == addr) 
        { 
            if (xtime.tv_sec - wind->date >= LIFETIME_WIND) 
            { 
                prev->next = wind->next; 
                if (wind == tail_windlist) /* Are we deleting the tail?    */ 
                    tail_windlist = prev; 
                kfree(wind); 
                counter_windlist--; 
                return NULL; 
            } 
            else 
                return wind; 
        } 
  
        prev = wind; 
        wind = wind->next; 
    } 
  
    return NULL; 
} 
  
/* 
 *  AddWind() function 
 *  Add a new element to the window sizes linked list. 
 */ 

void AddWind(__u32 addr, unsigned short window) 
{ 
    PTWIND wind, tmp = NULL; 
  
    wind = SeekWind(addr);               /* Already in memory?              */ 
  
    if (!wind)                           /* If not...                       */ 
    { 
        /* If no more room is available then try to make some.              */ 
        if (counter_windlist > MAX_WINDLIST) 

            GarbCollWindList(); 
  
        /* If it didn't work then don't store this entry.                   */ 
        if (counter_windlist > MAX_WINDLIST) 
            return; 
  
        tmp = (PTWIND) kmalloc(sizeof(WIND), GFP_KERNEL); 
  
        if (tmp) 
        { 
            tmp->addr   = addr; 
            tmp->window = window; 
            tmp->date   = xtime.tv_sec;  /* Current time stamp              */ 
            tmp->next   = NULL; 
            tail_windlist->next = tmp; 
            tail_windlist = tmp; 
            counter_windlist++; 
        } 
        else 
            printk("AddWind(): Out of memory."); 
            /* System crash? */ 
    } 
    else                                 /* If it's already there...        */ 
    { 
        wind->window = max(wind->window, window);  /* ...update window size */ 
        wind->date   = xtime.tv_sec;     /* Current time stamp              */ 
    } 
    return; 
} 
  
/* 
 *  KillWindList() function 
 *  Kills the window sized linked list. 
 */ 
  
void KillWindList(void) 
{ 
    PTWIND tmp, wind = head_windlist; 
  
    while (wind) 
    { 
        tmp = wind->next; 
        kfree(wind); 
        wind = tmp; 
    } 
  
    head_windlist = tail_windlist = NULL; 
} 
  
/* 
 *  GarbCollWindList() function 
 *  Deletes all the outdated entries in the linked list. 
 */ 

void GarbCollWindList(void) 
{ 
    PTWIND wind, prev; 
  
    wind = prev = head_windlist; 
  
    while (wind) 
    { 
        if ((xtime.tv_sec - wind->date >= LIFETIME_WIND) 
            && (wind->addr != HEAD_LIST_ADDR)) 
        { 
            prev->next = wind->next; 
            if (wind == tail_windlist) /* Are we deleting the tail?    */ 
                tail_windlist = prev; 
            kfree(wind); 
            counter_windlist--; 
            wind = prev->next; 
        } 
        else 
        { 
            prev = wind; 
            wind = wind->next; 
        } 
    } 
} 

/* 
 *  CollectWind() function 
 *  Adds a window size to the linked list provided it is 
 *  a valide one. 
 */ 
  
void CollectWind(struct sock *sk) 
{ 
    /* Add a window size to the linked list */ 
    if (head_windlist && sk->daddr && sk->cong_window) 
        AddWind(sk->daddr, sk->cong_window); 
} 
  
/* 
 *  WindManager() function 
 *  Manages the beginning and the ending of the process 
 *  of collecting window sizes, plus displaying the list. 
 */ 
  
void WindManager(struct sock *sk) 
{ 

    if ((sk->daddr == in_aton("128.197.10.100")) && (!head_windlist)) 
    { 
        printk("Ref. to cs1 detected: windows collection started\n"); 
        tail_windlist = head_windlist = NewWindList(); 
    } 

    if ((sk->daddr == in_aton("128.197.10.101")) && (head_windlist)) 
    { 
        printk("Ref. to cs2 detected: Killing windows list\n"); 
        KillWindList(); 
    } 
}

6.1.3 Modifications in the existing code

In linux/net/ipv4/tcp.c:

In tcp_connect(), after release_sock(sk);, added WindManager(sk);

In linux/net/ipv4/tcp_intput.c:

if (head_windlist)
{ 
    wind = SeekWind(saddr);
    if (wind) newsk->cong_window = wind->window;
}

6.2 Code for the extra feature that displays captured window sizes

In linux/include/linux/proc_fs.h:

In linux/fs/proc/root.c:

proc_register(&proc_root, &(struct proc_dir_entry) {
        PROC_PROJECT, 7, "project",
        S_IFREG | S_IRUGO, 1, 0, 0,
});

In linux/fs/proc/array.c:

and an extra case in get_root_array():

case PROC_PROJECT:
        return get_project_status(page);

In linux/net/ipv4/tcp.c:

/* 
 *  get_project_status() function 
 *  Invoked when a user process opens /proc/project. 
 *  Displays the current list of window sizes. 
 */ 
  
extern int get_project_status(char * page) 
{ 
    PTWIND wind = head_windlist; 
    int    len  = 0; 
  
    while (wind) 
    { 
        if (wind->addr != HEAD_LIST_ADDR) 
        { 
            len += sprintf(page + len, "%s: cong_window = %u for %u secs\n", 
                 in_ntoa(wind->addr), wind->window, xtime.tv_sec - wind->date); 
        } 
        wind = wind->next; 
    } 
    return len; 
}

6.3 Code for the implementation of delays on the client side

In linux/include/net/tcp.h, added:

#define ACK_DELAY HZ/50         /* This amounts to a delay of 20 ms */
extern void tcp_real_send_ack(struct sock *sk);

In linux/net/ipv4/tcp_input.c:

In linux/net/ipv4/tcp_output.c:

defined as follows:

void tcp_send_ack(struct sock *sk)
{
    tcp_send_delayed_ack(sk, ACK_DELAY, ACK_DELAY);
}

In linux/net/ipv4/tcp_timer.c:

6.4 HTTP simulation and conv

6.4.1 Server side: httpsims

/* ------------------------------------------------------------------------ */ 
/*  Olivier Hartmann - odh@bu.edu                                           */ 
/*  httpsims.c - 23 Sept 97                                                 */ 
/* ------------------------------------------------------------------------ */ 
/*  Simulates an HTTP server by sending <sizetransf> bytes to a client and  */ 
/*  closing the connection.  Use in conjunction with httpsimc.c.            */ 
/* ------------------------------------------------------------------------ */ 

#include <stdio.h> 
#include <stdlib.h> 
#include <unistd.h> 
#include <signal.h> 
#include <netdb.h>*/ 
#include <time.h> 
#include <sys/types.h> 
#include <sys/socket.h> 
#include <sys/wait.h> 
#include <sys/resource.h> 
#include <netinet/in.h> 

#define N_PORT      4200     /* Fixed server port number          */ 
#define EMBUFSIZE  65536     /* Emission buffer size              */ 

void EndChild(int sig); 
void Service(int sock); 

char embuf[EMBUFSIZE];       /* Emission buffer                   */ 
long sizetransf;             /* How many bytes to send to clients */ 
char sizetransfchar[20]; 

void main(int argc, char *argv[]) 
{ 
    int  sock, sock_a;       /* Listening socket; service socket  */ 
    struct sockaddr_in serv; /* Socket address                    */ 
    int  lg;                 /* Socket address size               */ 
    char nom[80];            /* Local host name                   */ 
    int  pid, i; 

    if (argc < 2) 
    { 
        fprintf(stderr, 
           "%s <sizetransf>\n", argv[0]); 
        exit(1); 
    } 

    sizetransf = atoi(argv[1]); 
    if (sizetransf <= 0) 
    { 
        perror("Invalid sizetransf"); 
        exit(1); 
    } 
    else 
        strcpy(sizetransfchar, argv[1]); 

    /* Initializes the pseudo-random number generator             */ 
    srand((unsigned int)time(NULL) / getpid()); 
  
    /* Fill in the emission buffer with some random sequence      */ 
    for (i=0; i < EMBUFSIZE; i++) 
        embuf[i] = rand() % 256; 

    /* Open listening socket                                      */ 
    if ((sock = socket(AF_INET, SOCK_STREAM, 0)) < 0) 
    { 
        perror("Cannot open a socket"); 
        exit(1); 
    } 

    /* Build listening socket address structure                             */ 
    serv.sin_family      = AF_INET;           /* IP domain                  */ 
    serv.sin_addr.s_addr = htonl(INADDR_ANY); /* Multiples addresses        */ 
    serv.sin_port        = htons(N_PORT);     /* fixed port number          */ 

    /* Bind listening socket                                                */ 
    if (bind(sock, (struct sockaddr *) &serv, sizeof(serv)) == -1) 
    { 
        perror("Cannot bind the socket"); 
        exit(1); 
    } 
     gethostname(nom, 80); 
    printf("\nServer running on %s on port %d.\n", nom, N_PORT); 

    /* Launch SIGCHLD handler                                               */ 
    signal(SIGCHLD, (void (*)())EndChild); 

    /* Launch connection queue                                              */ 
    listen(sock, 10);         /* 10 clients max                             */ 

    for (;;)                  /* Server loop                                */ 
    { 
        lg = sizeof(serv); 
        do 
            sock_a = accept(sock, (struct sockaddr *) &serv, &lg); 
        while (sock_a == -1); 
        pid = fork(); 
        if (pid == 0) 
        {                     /* Child process                              */ 
            close(sock);      /* Close the listening socket                 */ 
            Service(sock_a);  /* Start the service                          */ 
        } 
        else 
        {                     /* Parent process                             */ 
            close(sock_a);    /* Close the service socket                   */ 
        } 
    } 
} 

/* ------------------------------------------------------------------------ */ 
/*  Detects and processes a child process actual termination.               */ 
/* ------------------------------------------------------------------------ */ 
void EndChild(int sig) 
{ 
    while (wait3(NULL, WNOHANG, NULL) > 0);     /* We don't want zombies    */ 
    signal(SIGCHLD, (void (*)())EndChild);      /* Relaunch SIGCHLD handler */ 
} 

/* ------------------------------------------------------------------------ */ 
/*  Service function                                                        */ 
/*  The server just sends <sizetransf> bytes and closes the connection.     */ 
/* ------------------------------------------------------------------------ */ 
void Service(int sock) 
{ 
    char sizechar[80] = "";    /* How many bytes in a string  */ 
    char sizeofsize;           /* Size of that string         */ 
    int  sendcount = 0; 
    int i; 

    /* Send transfer size.  Protocol: the size as a string that follows */ 
    /* the size of that string (for portability). */ 
    sizeofsize = strlen(sizetransfchar); 
    send(sock, (char *) &sizeofsize, sizeof(sizeofsize), 0); 
    send(sock, sizetransfchar, sizeofsize, 0); 

    /* Send data in EMBUFSIZE chunks */ 
    for(i = 0; i < (sizetransf / EMBUFSIZE); i++) 
        sendcount += send(sock, embuf, EMBUFSIZE, 0); 
    sendcount += send(sock, embuf, (int)(sizetransf % EMBUFSIZE), 0); 

    exit(0); 
}

6.4.2 Client side: httpsimc

/* ------------------------------------------------------------------------ */ 
/*  Olivier Hartmann - odh@bu.edu                                           */ 
/*  httpsimc.c - 23 Sept 97                                                 */ 
/* ------------------------------------------------------------------------ */ 
/*  Used in conjunction with httpsims.c, simulates an HTTP client by        */ 
/*  opening a TCP connection with the server, receiving data, closing the   */ 
/*  connection and starting all over again for <duration> seconds.          */ 
/* ------------------------------------------------------------------------ */ 

#include <stdio.h> 
#include <string.h> 
#include <netdb.h> 
#include <sys/time.h> 
#include <sys/types.h> 
#include <sys/socket.h> 
#include <netinet/in.h> 

#define RECBUFSIZE 1024      /* Reception buffer size             */ 

void main(int argc, char *argv[]) 
{ 
    int  sizetransf;           /* How many bytes to receive        */ 
    char sizechar[80];         /* How many bytes in a string       */ 
    char sizeofsize; 
    int  sock;                 /* Server connection socket         */ 
    struct sockaddr_in serv;   /* Server connection socket address */ 
    struct hostent *as;        /* Server IP address structure      */ 
    int  i; 

    char recbuf[RECBUFSIZE];   /* Reception buffer                 */ 
    long diff;                 /* How many bytes missed            */ 
    int  reccount;             /* Amount of data received          */ 
    long startclock, endclock; /* Transmission time variables      */ 
    int expduration;           /* Duration of emission             */ 
    float duration = 0;        /* Duration counter                 */ 
    struct timeval tv; 
    struct timezone tz; 

    if (argc < 4) 
    { 
        fprintf(stderr, 
           "%s <machine> <port> <duration>\n", argv[0]); 
        exit(1); 
    } 

    expduration = atoi(argv[3]); 
    if (expduration <= 0) 
    { 
        perror("Invalid duration"); 
        exit(1); 
    } 

    if (!(as = gethostbyname(argv[1]))) /* Server IP address      */ 
    { 
        perror("gethostbyname() error"); 
        exit(1); 
    } 

    /* Build server address structure                             */ 
    serv.sin_family = AF_INET; 
    serv.sin_port   = htons(atoi(argv[2])); 
    memcpy((char *) &serv.sin_addr, as->h_addr_list[0], as->h_length); 

    printf("Receiving... "); 
    fflush(NULL); 

    gettimeofday(&tv, &tz); /* Computes time to 1/100th second \/ */ 
    startclock = (tv.tv_sec & 0xffffff)*100 + tv.tv_usec/10000; 

    while (duration < expduration * 100) /* For just <duration> seconds... */ 
    { 
        reccount = 0; 

        /* Open socket */ 
        if ((sock = socket(AF_INET, SOCK_STREAM, 0)) < 0) 
        { 
            perror("Cannot open a socket"); 
            exit(1); 
        } 

        /* Connect to server */ 
        if (connect(sock, (struct sockaddr *) &serv, sizeof(serv)) == -1) 
        { 
            perror("Cannot connect to the server"); 
            exit(1); 
        } 

        /* Gets the amount of bytes to receive.  Protocol: the amount as a */ 
        /* string that follows the size of that string (for portability). */ 
        recv(sock, (char *) &sizeofsize, sizeof(sizeofsize), 0); 
        recv(sock, sizechar, sizeofsize, 0); 
        sizetransf = atoi(sizechar); 

        if (sizetransf > 0) 
        { 
            for(i = 0; i < (sizetransf / RECBUFSIZE); i++) 
                reccount += recv(sock, recbuf, RECBUFSIZE, 0); 
            reccount += recv(sock, recbuf, (int)(sizetransf % RECBUFSIZE), 0); 
  
            /* In case we missed a few bytes (that happens)... */ 
            diff = sizetransf - reccount; 
            if (diff != 0) 
            { 
                for(i = 0; i < (diff / RECBUFSIZE); i++) 
                    reccount += recv(sock, recbuf, RECBUFSIZE, 0); 
                reccount += recv(sock, recbuf, (int)(diff % RECBUFSIZE), 0); 
            } 
        } 
        else 
        { 
            perror("Invalid data length"); 
            exit(1); 
        } 

        close(sock); /* Closes the connection */ 

        gettimeofday(&tv, &tz); /* Computes time to 1/100th second \/ */ 
        endclock = (tv.tv_sec & 0xffffff) * 100 + tv.tv_usec/10000; 
        duration = (endclock - startclock); 
  
    } 
    printf(" done.\n"); 
}

6.4.3 conv

/* ------------------------------------------------------------------------ */ 
/*  Olivier Hartmann - odh@bu.edu                                           */ 
/*  conv.c - 5 Oct 97                                                       */ 
/* ------------------------------------------------------------------------ */ 

#include <stdio.h> 
#include <stdlib.h> 
#include <string.h> 
#include <unistd.h> 

#define LINE_S_MAX   255  /* Line size maximum                */ 
#define DATE_S_MAX    10  /* Date size maximum                */ 
#define SEQ_S_MAX     12  /* TCP sequence number size maximum */ 

#define min(x, y) ((x < y) ? x : y) 

typedef struct                       /* A text file  line     */ 
{ 
    char date[DATE_S_MAX+1];         /* Date                  */ 
    char seq[SEQ_S_MAX+1];           /* TCP sequence number   */ 
} LINE; 

void ProcLine(char *line); 
LINE FormatLine(char *line); 
void TraiteCmd(LINE a_line); 
void Err(char *msg); 

FILE    *f_in;                       /* File to process        */ 
FILE    *f_out;                      /* File to ouput          */ 
char    line[LINE_S_MAX+1];          /* One text line          */ 
int     linenum = 1;                 /* Number of current line */ 
double  basedate; 
int     baseseq; 
int     countseq; 
int     prevseq; 
int     addcount = 0; 

int main(void) 
{ 
    if (!(f_in = fdopen(STDIN_FILENO, "r"))) 
        Err("Can't open STDIN!\n"); 
    if (!(f_out = fdopen(STDOUT_FILENO, "w"))) 
        Err("Can't open STDOUT!\n"); 

    while(!feof(f_in)) 
        if (fgets(line, LINE_S_MAX, f_in)) /* Reads one line             */ 
        { 
            line[ strlen(line)-1 ] = '\0'; /* Gets rid of the final LF   */ 
            if (line[0] != '\0') 
                ProcLine(line);            /* Processes a non-empty line */ 
            linenum++; 
        } 

    fclose(f_out); 
    fclose(f_in); 
} 

/* ------------------------------------------------------------------------ */ 
/*  Processes a text line                                                   */ 
/* ------------------------------------------------------------------------ */ 
void ProcLine(char *line) 
{ 
    LINE   a_line; 
    double date; 
    int    seq; 

    a_line = FormatLine(line); 

    date = atof(a_line.date); 
    seq  = atoi(a_line.seq); 

    if (linenum == 1)                /* First line: sets up bases        */ 
    { 
        basedate = date; 
        baseseq  = seq; 
    } 

    if (abs(seq - prevseq) > 10000)  /* Detects a new sequence           */ 
    { 
        baseseq = seq; 
        addcount += countseq; 
    } 

    prevseq = seq;                   /* Saves previous sequence number   */ 
    seq  -= baseseq; 
    date -= basedate; 
    countseq = seq;                  /* Keeps track of the current count */ 

    fprintf(f_out, "%.2f %f\n", date, (double)(seq + addcount)/1024); 
} 

/* ------------------------------------------------------------------------ */ 
/*  Turns a text line into its date and seq fields                          */ 
/* ------------------------------------------------------------------------ */ 
LINE FormatLine(char *line) 
{ 
    LINE a_line; 
    int  j, i = 0; 

    a_line.date[0] = '\0'; 
    a_line.seq[0]  = '\0'; 

    while(line[i] != ' ' && line[i] != '\0') i++; 

    if (line[i] == ' ') 
    { 
        memcpy(a_line.date, line, min(i, DATE_S_MAX)); 
        a_line.date[ min(i, DATE_S_MAX) ] = '\0'; 

        j = strlen(line) - 1; 
        if (i < j) 
        { 
            memcpy(a_line.seq, line+i+3, min(j-i+1, SEQ_S_MAX)); 
            a_line.seq[ min(j-i+1, SEQ_S_MAX) ] = '\0'; 
        } 
    } 
    return(a_line); 
} 

/* ------------------------------------------------------------------------ */ 
/* Error handling and exit                                                  */ 
/* ------------------------------------------------------------------------ */ 
void Err(char *msg) 
{ 
    printf("Error: %s\n", msg); 
    fclose(f_out); 
    fclose(f_in); 
    exit(0); 
}