|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||
java.lang.Objectorg.apache.hadoop.mapreduce.InputSplit
org.apache.hadoop.mapreduce.lib.input.CombineFileSplit
@InterfaceAudience.Public @InterfaceStability.Stable public class CombineFileSplit
A sub-collection of input files.
Unlike FileSplit, CombineFileSplit class does not represent
a split of a file, but a split of input files into smaller sets.
A split may contain blocks from different file but all
the blocks in the same split are probably local to some rack
CombineFileSplit can be used to implement RecordReader's,
with reading one record per file.
FileSplit,
CombineFileInputFormat| Constructor Summary | |
|---|---|
CombineFileSplit()
default constructor |
|
CombineFileSplit(CombineFileSplit old)
Copy constructor |
|
CombineFileSplit(Path[] files,
long[] lengths)
|
|
CombineFileSplit(Path[] files,
long[] start,
long[] lengths,
String[] locations)
|
|
| Method Summary | |
|---|---|
long |
getLength()
Get the size of the split, so that the input splits can be sorted by size. |
long |
getLength(int i)
Returns the length of the ith Path |
long[] |
getLengths()
Returns an array containing the lengths of the files in the split |
String[] |
getLocations()
Returns all the Paths where this input-split resides |
int |
getNumPaths()
Returns the number of Paths in the split |
long |
getOffset(int i)
Returns the start offset of the ith Path |
Path |
getPath(int i)
Returns the ith Path |
Path[] |
getPaths()
Returns all the Paths in the split |
long[] |
getStartOffsets()
Returns an array containing the start offsets of the files in the split |
void |
readFields(DataInput in)
Deserialize the fields of this object from in. |
String |
toString()
|
void |
write(DataOutput out)
Serialize the fields of this object to out. |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait |
| Constructor Detail |
|---|
public CombineFileSplit()
public CombineFileSplit(Path[] files,
long[] start,
long[] lengths,
String[] locations)
public CombineFileSplit(Path[] files,
long[] lengths)
public CombineFileSplit(CombineFileSplit old)
throws IOException
IOException| Method Detail |
|---|
public long getLength()
InputSplit
getLength in class InputSplitpublic long[] getStartOffsets()
public long[] getLengths()
public long getOffset(int i)
public long getLength(int i)
public int getNumPaths()
public Path getPath(int i)
public Path[] getPaths()
public String[] getLocations()
throws IOException
getLocations in class InputSplitIOException
public void readFields(DataInput in)
throws IOException
Writablein.
For efficiency, implementations should attempt to re-use storage in the existing object where possible.
readFields in interface Writablein - DataInput to deseriablize this object from.
IOException
public void write(DataOutput out)
throws IOException
Writableout.
write in interface Writableout - DataOuput to serialize this object into.
IOExceptionpublic String toString()
toString in class Object
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||