Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## org.apache.hadoop.mapreduce.lib.join
Class CompositeRecordReader<K extends WritableComparable<?>,V extends Writable,X extends Writable>

    java.lang.Object
      org.apache.hadoop.mapreduce.RecordReader<K,V>
          org.apache.hadoop.mapreduce.lib.join.ComposableRecordReader<K,X>
              org.apache.hadoop.mapreduce.lib.join.CompositeRecordReader<K,V,X>

All Implemented Interfaces:
  Closeable, Comparable<ComposableRecordReader<K,?>>, Configurable

Direct Known Subclasses:
  JoinRecordReader, MultiFilterRecordReader

    @InterfaceAudience.Public
    @InterfaceStability.Stable
    public abstract class CompositeRecordReader<K extends WritableComparable<?>,V extends Writable,X extends Writable>

extends ComposableRecordReader<K,X>
implements Configurable

A RecordReader that can effect joins of RecordReaders sharing a common key type and partitioning.

Field Summary
`protected Configuration` | `conf`
`protected org.apache.hadoop.mapreduce.lib.join.CompositeRecordReader.JoinCollector` | `jc`
`protected K` | `key`
`protected Class<? extends WritableComparable>` | `keyclass`
`protected ComposableRecordReader<K,? extends V>[]` | `kids`
`protected X` | `value`

Constructor Summary
`CompositeRecordReader(int id, int capacity, Class<? extends WritableComparator> cmpcl)` Create a RecordReader with `capacity` children to position `id` in the parent reader.

Method Summary
`void` | `accept(org.apache.hadoop.mapreduce.lib.join.CompositeRecordReader.JoinCollector jc, K key)` If key provided matches that of this Composite, give JoinCollector iterator over values it may emit.
`void` | `add(ComposableRecordReader<K,? extends V> rr)` Add a RecordReader to this collection.
`void` | `close()` Close all child RRs.
`protected abstract boolean` | `combine(Object[] srcs, TupleWritable value)`
`int` | `compareTo(ComposableRecordReader<K,?> other)` Implement Comparable contract (compare key of join or head of heap with that of another).
`protected K` | `createKey()` Create a new key common to all child RRs.
`protected TupleWritable` | `createTupleWritable()` Create a value to be used internally for joins.
`protected void` | `fillJoinCollector(K iterkey)` For all child RRs offering the key provided, obtain an iterator at that position in the JoinCollector.
`protected WritableComparator` | `getComparator()` Return comparator defining the ordering for RecordReaders in this composite.
`Configuration` | `getConf()` Return the configuration used by this object.
`K` | `getCurrentKey()` Get the current key
`X` | `getCurrentValue()` Get the current value.
`protected abstract ResetableIterator<X>` | `getDelegate()` Obtain an iterator over the child RRs apropos of the value type ultimately emitted from this join.
`float` | `getProgress()` Report progress as the minimum of all child RR progress.
`protected PriorityQueue<ComposableRecordReader<K,?>>` | `getRecordReaderQueue()` Return sorted list of RecordReaders for this composite.
`boolean` | `hasNext()` Return true if it is possible that this could emit more values.
`int` | `id()` Return the position in the collector this class occupies.
`void` | `initialize(InputSplit split, TaskAttemptContext context)` Called once at initialization.
`K` | `key()` Return the key for the current join or the value at the top of the RecordReader heap.
`void` | `key(K key)` Clone the key at the top of this RR into the given object.
`void` | `setConf(Configuration conf)` Set the configuration to be used by this object.
`void` | `skip(K key)` Pass skip key to child RRs.

Methods inherited from class org.apache.hadoop.mapreduce.RecordReader
`nextKeyValue`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Field Detail

### conf

    protected Configuration conf

### keyclass

    protected Class<? extends WritableComparable> keyclass

### jc

    protected final org.apache.hadoop.mapreduce.lib.join.CompositeRecordReader.JoinCollector jc

### kids

    protected final ComposableRecordReader<K extends WritableComparable<?>,? extends V extends Writable>[] kids

### key

    protected K extends WritableComparable<?> key

### value

    protected X extends Writable value

Constructor Detail

### CompositeRecordReader

    public CompositeRecordReader(int id,
                                 int capacity,
                                 Class<? extends WritableComparator> cmpcl)
                          throws IOException

  Create a RecordReader with `capacity` children to position `id` in the parent reader. The id of a root CompositeRecordReader is -1 by convention, but relying on this is not recommended.

  Throws:
    `IOException`

Method Detail

### combine

    protected abstract boolean combine(Object[] srcs,
                                       TupleWritable value)

### initialize

    public void initialize(InputSplit split,
                           TaskAttemptContext context)
                    throws IOException,
                           InterruptedException

  Description copied from class: `RecordReader`
  Called once at initialization.

  Specified by:
    `initialize` in class `RecordReader<K extends WritableComparable<?>,X extends Writable>`
  Parameters:
    `split` - the split that defines the range of records to read
    `context` - the information about the task
  Throws:
    `IOException`
    `InterruptedException`

### id

    public int id()

  Return the position in the collector this class occupies.

### setConf

    public void setConf(Configuration conf)

  Set the configuration to be used by this object.

  Specified by:
    `setConf` in interface `Configurable`

### getConf

    public Configuration getConf()

  Return the configuration used by this object.

  Specified by:
    `getConf` in interface `Configurable`

### getRecordReaderQueue

    protected PriorityQueue<ComposableRecordReader<K,?>> getRecordReaderQueue()

  Return sorted list of RecordReaders for this composite.

### getComparator

    protected WritableComparator getComparator()

  Return comparator defining the ordering for RecordReaders in this composite.

### add

    public void add(ComposableRecordReader<K,? extends V> rr)
             throws IOException,
                    InterruptedException

  Add a RecordReader to this collection. The id() of a RecordReader determines where in the Tuple its entry will appear. Adding RecordReaders with the same id has undefined behavior.

  Throws:
    `IOException`
    `InterruptedException`

### key

    public K key()

  Return the key for the current join or the value at the top of the RecordReader heap.

### key

    public void key(K key)
             throws IOException

  Clone the key at the top of this RR into the given object.

  Throws:
    `IOException`

### getCurrentKey

    public K getCurrentKey()

  Description copied from class: `RecordReader`
  Get the current key

  Specified by:
    `getCurrentKey` in class `RecordReader<K extends WritableComparable<?>,X extends Writable>`
  Returns:
    the current key or null if there is no current key

### hasNext

    public boolean hasNext()

  Return true if it is possible that this could emit more values.

### skip

    public void skip(K key)
              throws IOException,
                     InterruptedException

  Pass skip key to child RRs.

  Throws:
    `IOException`
    `InterruptedException`

### getDelegate

    protected abstract ResetableIterator<X> getDelegate()

  Obtain an iterator over the child RRs apropos of the value type ultimately emitted from this join.

### accept

    public void accept(org.apache.hadoop.mapreduce.lib.join.CompositeRecordReader.JoinCollector jc,
                       K key)
                throws IOException,
                       InterruptedException

  If key provided matches that of this Composite, give JoinCollector iterator over values it may emit.

  Throws:
    `IOException`
    `InterruptedException`

### fillJoinCollector

    protected void fillJoinCollector(K iterkey)
                              throws IOException,
                                     InterruptedException

  For all child RRs offering the key provided, obtain an iterator at that position in the JoinCollector.

  Throws:
    `IOException`
    `InterruptedException`

### compareTo

    public int compareTo(ComposableRecordReader<K,?> other)

  Implement Comparable contract (compare key of join or head of heap with that of another).

  Specified by:
    `compareTo` in interface `Comparable<ComposableRecordReader<K extends WritableComparable<?>,?>>`

### createKey

    protected K createKey()

  Create a new key common to all child RRs.

  Throws:
    `ClassCastException` - if key classes differ.

### createTupleWritable

    protected TupleWritable createTupleWritable()

  Create a value to be used internally for joins.

### getCurrentValue

    public X getCurrentValue()
                                       throws IOException,
                                              InterruptedException

  Get the current value.

  Specified by:
    `getCurrentValue` in class `RecordReader<K extends WritableComparable<?>,X extends Writable>`
  Returns:
    the object that was read
  Throws:
    `IOException`
    `InterruptedException`

### close

    public void close()
               throws IOException

  Close all child RRs.

  Specified by:
    `close` in interface `Closeable`
  Specified by:
    `close` in class `RecordReader<K extends WritableComparable<?>,X extends Writable>`
  Throws:
    `IOException`

### getProgress

    public float getProgress()
                      throws IOException,
                             InterruptedException

  Report progress as the minimum of all child RR progress.

  Specified by:
    `getProgress` in class `RecordReader<K extends WritableComparable<?>,X extends Writable>`
  Returns:
    a number between 0.0 and 1.0 that is the fraction of the data read
  Throws:
    `IOException`
    `InterruptedException`

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2012 Apache Software Foundation. All Rights Reserved.