Obtener rutas de entrada totales en Hadoop Reducer

I still cannot figure out how to get the total number of input paths in hadoop. I've saw Obtenga el recuento total de rutas de entrada en Hadoop Mapper, but don't understand. I just want to know that, how many files in the input path...Any help, please?

preguntado el 12 de junio de 12 a las 17:06

1 Respuestas

Like i said towards the end of my answer in the referenced question, if you're using v0.20.203+ of hadoop, then you can query the configuration property mapreduce.input.num.files

long inputFiles = context.getConfiguration().getLong('mapreduce.input.num.files');

Respondido el 12 de junio de 12 a las 17:06

But, why I always get 1?int N = (int) context.getConfiguration().getLong("mapreduce.input.num.files", 1); It seems that such a property doesn't exists? - Kinka

¿Qué versión de hadoop estás usando? - Chris White

It is v0.20.2, is it the problem? - Kinka

0.20.2 is not 0.20.203+, that's why it doesn't work. If you are using files which are compressed with gzip and an input format such as TextInputFormat, then mapred.map.tasks will most probably give you the number of input files. - Chris White

No es la respuesta que estás buscando? Examinar otras preguntas etiquetadas or haz tu propia pregunta.