|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||
java.lang.Objectorg.apache.hadoop.mapred.lib.CombineFileSplit
public class CombineFileSplit
A sub-collection of input files. Unlike FileSplit,
CombineFileSplit * class does not represent a split of a file, but a split of input files
into smaller sets. A split may contain blocks from different file but all
the blocks in the same split are probably local to some rack
CombineFileSplit can be used to implement RecordReader's,
with reading one record per file.
FileSplit,
CombineFileInputFormat| Constructor Summary | |
|---|---|
CombineFileSplit()
default constructor |
|
CombineFileSplit(CombineFileSplit old)
Copy constructor |
|
CombineFileSplit(JobConf job,
Path[] files,
long[] lengths)
|
|
CombineFileSplit(JobConf job,
Path[] files,
long[] start,
long[] lengths,
String[] locations)
|
|
| Method Summary | |
|---|---|
JobConf |
getJob()
|
long |
getLength()
Get the total number of bytes in the data of the InputSplit. |
long |
getLength(int i)
Returns the length of the ith Path |
long[] |
getLengths()
Returns an array containing the lengths of the files in the split |
String[] |
getLocations()
Returns all the Paths where this input-split resides |
int |
getNumPaths()
Returns the number of Paths in the split |
long |
getOffset(int i)
Returns the start offset of the ith Path |
Path |
getPath(int i)
Returns the ith Path |
Path[] |
getPaths()
Returns all the Paths in the split |
long[] |
getStartOffsets()
Returns an array containing the startoffsets of the files in the split |
void |
readFields(DataInput in)
Deserialize the fields of this object from in. |
String |
toString()
|
void |
write(DataOutput out)
Serialize the fields of this object to out. |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait |
| Constructor Detail |
|---|
public CombineFileSplit()
public CombineFileSplit(JobConf job,
Path[] files,
long[] start,
long[] lengths,
String[] locations)
public CombineFileSplit(JobConf job,
Path[] files,
long[] lengths)
public CombineFileSplit(CombineFileSplit old)
throws IOException
IOException| Method Detail |
|---|
public JobConf getJob()
public long getLength()
InputSplitInputSplit.
getLength in interface InputSplitpublic long[] getStartOffsets()
public long[] getLengths()
public long getOffset(int i)
public long getLength(int i)
public int getNumPaths()
public Path getPath(int i)
public Path[] getPaths()
public String[] getLocations()
throws IOException
getLocations in interface InputSplitInputSplit is
located as an array of Strings.
IOException
public void readFields(DataInput in)
throws IOException
Writablein.
For efficiency, implementations should attempt to re-use storage in the existing object where possible.
readFields in interface Writablein - DataInput to deseriablize this object from.
IOException
public void write(DataOutput out)
throws IOException
Writableout.
write in interface Writableout - DataOuput to serialize this object into.
IOExceptionpublic String toString()
toString in class Object
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||