org.apache.nutch.crawl
Class Generator.Selector

java.lang.Object
  org.apache.nutch.crawl.Generator.Selector

All Implemented Interfaces:

Closeable, JobConfigurable, Mapper, Partitioner, Reducer

Enclosing class:

Generator

public static class Generator.Selector

extends Object

implements Mapper, Partitioner, Reducer

Selects entries due for fetch.

Constructor Summary

Generator.Selector()

Method Summary

void	close()
void	configure(JobConf job)
int	getPartition(WritableComparable key, Writable value, int numReduceTasks) Partition by host.
void	map(WritableComparable key, Writable value, OutputCollector output, Reporter reporter) Select & invert subset due for fetch.
void	reduce(WritableComparable key, Iterator values, OutputCollector output, Reporter reporter) Collect until limit is reached.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Generator.Selector

public Generator.Selector()

Method Detail

configure

public void configure(JobConf job)

Specified by:

configure in interface JobConfigurable

close

public void close()

Specified by:

close in interface Closeable

map

public void map(WritableComparable key,
                Writable value,
                OutputCollector output,
                Reporter reporter)
         throws IOException

Select & invert subset due for fetch.

Specified by:

map in interface Mapper

Throws:

IOException

getPartition

public int getPartition(WritableComparable key,
                        Writable value,
                        int numReduceTasks)

Partition by host.

Specified by:

getPartition in interface Partitioner

reduce

public void reduce(WritableComparable key,
                   Iterator values,
                   OutputCollector output,
                   Reporter reporter)
            throws IOException

Collect until limit is reached.

Specified by:

reduce in interface Reducer

Throws:

IOException