Showing posts with label MapReduce. Show all posts
Showing posts with label MapReduce. Show all posts

Saturday, October 21, 2017

Exercises to study Java Stream API


In this blog post, we will study Stream API of Java 9 with several exercise questions. In these exercises, we will use two different domains: 
  • IMDB Movies
  • World Countries
Before we dive into the exercise questions, let's take a look at these domains:

IMDB Movies: We have three domain classes: Movie, Director and Genre. Each movie has a title, a year and a unique id assigned by Internet Movie DataBase (IMDB). Each movie has been directed by at least one director, and each director has many movies. Each movie belongs to at least one genre.

package com.example.domain;

import java.util.ArrayList;
import java.util.List;

public class Movie {
private int id;
   private String title;
   private int year;
   private String imdb;
   private List<Genre> genres;
   private List<Director> directors;

   {
      genres = new ArrayList<>();
      directors = new ArrayList<>();
   }

   public Movie() {
   }

   public Movie(int id, String title, int year, String imdb) {
      this.id = id;
      this.title = title;
      this.year = year;
      this.imdb = imdb;
   }

   // getters and setters

   @Override   
   public String toString() {
      return "Movie [title=" + title + ", year=" + year + "]";
   }
}


package com.example.domain;
import java.util.ArrayList;
import java.util.List;

public class Director {
   private int id;
   private String name;
   private String imdb;
   private List<Movie> movies= new ArrayList<>();

   public Director() {
   }

   public Director(int id, String name, String imdb) {
      this.id = id;
      this.name = name;
      this.imdb = imdb;
   }

   // getters and setters
   @Override   
   public String toString() {
      return "Director [id=" + id + ", name=" + name + ", imdb=" + imdb + "]";
   }
}

package com.example.domain;
public class Genre {
   private int id;
   private String name;

   public Genre() {
   }

   public Genre(int id, String name) {
      this.id = id;
      this.name = name;
   }

   // getters and setters
@Override
   public String toString() {
      return "Genre [id=" + id + ", name=" + name + "]";
   }

}

World Countries: There are two domain classes: Country and City. Each city belongs to a country defined by the attribute, countryCode. Each country has a unique code and has many cities.

package com.example.domain;

import java.util.ArrayList;
import java.util.List;

public class Country {
   private String code;
   private String name;
   private String continent;
   private double surfaceArea;
   private int population;
   private double gnp;
   private int capital;
   private List<City> cities;
   {
      cities = new ArrayList<>();
   }

   public Country() {
   }

   public Country(String code, String name, String continent, int population,
         double surfaceArea, double gnp, int capital) {
      this.code = code;
      this.name = name;
      this.continent = continent;
      this.surfaceArea = surfaceArea;
      this.population = population;
      this.capital = capital;
      this.gnp = gnp;
   }

   // getters and setters
   @Override   
   public String toString() {
      return "Country [ name=" + name + ", population=" + population + "]";
   }

}

package com.example.domain;

public class City {
   private int id;
   private String name;
   private int population;
   private String countryCode;

   public City() {
   }

   public City(int id, String name, String countryCode, int population) {
      this.id = id;
      this.name = name;
      this.population = population;
      this.countryCode = countryCode;
   }

   // getters and setters
   
   @Override   
   public String toString() {
      return "City [id=" + id + ", name=" + name + ", population=" + population + ", countryCode=" + countryCode + "]";
   };

}

EXERCISE #1

Find the highest populated city of each country:

package com.example.exercise;

import java.util.Comparator;
import java.util.List;
import java.util.Optional;
import java.util.stream.Collectors;

import com.example.dao.CountryDao;
import com.example.dao.InMemoryWorldDao;
import com.example.domain.City;

public class Exercise1 {

   public static void main(String[] args) {
      CountryDao countryDao= InMemoryWorldDao.getInstance();
      List<City> highPopulatedCitiesOfCountries = countryDao.findAllCountries()
                .stream()
                .map( country -> country.getCities().stream().max(Comparator.comparing(City::getPopulation)))
                .filter(Optional::isPresent)
                .map(Optional::get)
                  .collect(Collectors.toList());
      highPopulatedCitiesOfCountries.forEach(System.out::println);
   }

}

The code given above will produce the following output:

City [id=3494, name=Auckland, population=381800, countryCode=NZL]
City [id=764, name=Suva, population=77366, countryCode=FJI]
City [id=2884, name=Port Moresby, population=247000, countryCode=PNG]
City [id=918, name=Les Abymes, population=62947, countryCode=GLP]
.
.
.
City [id=4067, name=Charlotte Amalie, population=13000, countryCode=VIR]
City [id=712, name=Cape Town, population=2352121, countryCode=ZAF]
City [id=538, name=Bandar Seri Begawan, population=21484, countryCode=BRN]
City [id=933, name=Tegucigalpa, population=813900, countryCode=HND]

EXERCISE #2

Find the most populated city of each continent:

package com.example.exercise;

import java.util.Comparator;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Optional;
import java.util.function.BiConsumer;
import java.util.function.Consumer;
import java.util.function.Predicate;
import java.util.stream.Collector;
import java.util.stream.Collectors;

import com.example.dao.CountryDao;
import com.example.dao.InMemoryWorldDao;
import com.example.domain.City;

public class Exercise2 {

    public static void main(String[] args) {
        CountryDao countryDao = InMemoryWorldDao.getInstance();
        final Predicate<Entry<String, Optional<City>>> isPresent = entry -> entry.getValue().isPresent();
        final BiConsumer<String, Optional<City>> printEntry =
                (k,v) -> {
                    City city = v.get();
                    System.out.println(k + ": City [ name= " + city.getName() + ", population= " + city.getPopulation() + " ]");
                };
        Collector<City, ?, Map<String, Optional<City>>> groupingHighPopulatedCitiesByContinent = Collectors.groupingBy(city -> countryDao.findCountryByCode(city.getCountryCode()).getContinent(), Collectors.maxBy(Comparator.comparing(City::getPopulation)));
        Map<String, Optional<City>> highPopulatedCitiesByContinent = countryDao.findAllCountries()
                .stream()
                .flatMap(country -> country.getCities().stream())
                .collect(groupingHighPopulatedCitiesByContinent);
        highPopulatedCitiesByContinent.forEach(printEntry);

    }

}

The code given above will produce the following output:

South America: City [ name= SÆo Paulo, population= 9968485 ]
Asia: City [ name= Mumbai (Bombay), population= 10500000 ]
Europe: City [ name= Moscow, population= 8389200 ]
Africa: City [ name= Cairo, population= 6789479 ]
North America: City [ name= Ciudad de M‚xico, population= 8591309 ]
Oceania: City [ name= Sydney, population= 3276207 ]


EXERCISE #3

Find the number of movies of each director: Try to solve this problem by assuming that Director class has not the member movies

package com.example.exercise;

import java.util.Collection;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;

import com.example.domain.Director;
import com.example.domain.Movie;
import com.example.service.InMemoryMovieService;
import com.example.service.MovieService;

public class Exercise3 {

    public static void main(String[] args) {
        MovieService movieService = InMemoryMovieService.getInstance();
        Collection<Movie> movies = movieService.findAllMovies();
        Map<String, Long> directorMovieCounts =
                movies.stream()
                        .map(Movie::getDirectors)
                        .flatMap(List::stream)
                        .collect(Collectors.groupingBy(Director::getName, Collectors.counting()));
        directorMovieCounts.entrySet().forEach(System.out::println);
    }

}


The code given above will produce the following output:

Sam Taylor Wood=1
F. Gary Gray=1
Oliver Hirschbiegel=1
Scott Cooper=1
Katherine Dieckmann=1
Kevin Macdonald=1
Peter Jackson=2
Andy Tennant=1
.
.
.
Isabel Coixet=1
Jason Reitman=1
Louie Psihoyos=1
Brian De Palma=1
Jay DiPietro=1

EXERCISE #4


Find the number of genres of each director's movies:

package com.example.exercise;

import java.util.Collection;
import java.util.Map;
import java.util.stream.Stream;

import com.example.domain.Director;
import com.example.domain.Genre;
import com.example.domain.Movie;
import com.example.service.InMemoryMovieService;
import com.example.service.MovieService;

import static java.util.stream.Collectors.*;

public class Exercise4 {

    public static void main(String[] args) {
        MovieService movieService = InMemoryMovieService.getInstance();
        Collection<Director> directors = movieService.findAllDirectors();
        Stream<DirectorGenre> stream =
                directors.stream()
                        .flatMap(director -> director.getMovies()
                                .stream()
                                .map(Movie::getGenres)
                                .flatMap(Collection::stream)
                                .map(genre -> new DirectorGenre(director, genre))
                                .collect(toList()).stream()
                        );
        Map<Director, Map<Genre, Long>> directorGenreList =
                stream.collect(
                        groupingBy(
                                DirectorGenre::getKey,
                                groupingBy(DirectorGenre::getValue, counting())
                        )
                );
        directorGenreList.forEach(
                (k1,v1) -> {
                    System.out.println(k1.getName());
                    v1.forEach( (k2,v2) -> {
                                System.out.println(String.format("\t%-12s: %2d", k2.getName(), v2));
                            });
                    System.out.println();
                }
        );
    }

}

class DirectorGenre implements Map.Entry<Director, Genre> {
    private Director director;
    private Genre genre;

    public DirectorGenre(Director director, Genre genre) {
        this.director = director;
        this.genre = genre;
    }

    @Override    public Director getKey() {
        return director;
    }

    @Override    public Genre getValue() {
        return genre;
    }

    @Override    public Genre setValue(Genre genre) {
        this.genre = genre;
        return genre;
    }

}

The code given above will produce the following output:

Marc Webb
 Comedy      :  1
 Drama       :  1
 Romance     :  1

Peter Hyams
 Drama       :  1
 Mystery     :  1

Mark Neveldine
 Action      :  1
 Sci-Fi      :  1
 Thriller    :  1

Brian Taylor
 Action      :  1
 Sci-Fi      :  1
 Thriller    :  1

.
.
.

Ryûhei Kitamura
 Fantasy     :  1
 Drama       :  1
 Action      :  1
 Thriller    :  1
 Adventure   :  1

Shusuke Kaneko
 Action      :  1

Gregor Jordan
 Drama       :  1
 Thriller    :  1

EXERCISE #5


Find the highest populated capital city:

package com.example.exercise;

import com.example.dao.CityDao;
import com.example.dao.CountryDao;
import com.example.dao.InMemoryWorldDao;
import com.example.domain.City;
import com.example.domain.Country;

import java.util.Objects;
import java.util.Optional;

import static java.lang.System.out;
import static java.util.Comparator.comparing;
import static java.util.stream.Collectors.maxBy;

public class Exercise5 {

    public static void main(String[] args) {
        CountryDao countryDao = InMemoryWorldDao.getInstance();
        CityDao cityDao = InMemoryWorldDao.getInstance();
        Optional<City> capital = countryDao.findAllCountries()
                .stream()
                .map(Country::getCapital)
                .map(cityDao::findCityById)
                .filter(Objects::nonNull)
                .collect(maxBy(comparing(City::getPopulation)));
        capital.ifPresent(out::println);
    }

}

The code given above will produce the following output:

City [id=2331, name=Seoul, population=9981619, countryCode=KOR]

EXERCISE #6


Find the highest populated capital city of each continent:

package com.example.exercise;

import com.example.dao.CityDao;
import com.example.dao.CountryDao;
import com.example.dao.InMemoryWorldDao;
import com.example.domain.City;

import java.util.Map;
import java.util.Optional;
import java.util.function.BiConsumer;
import java.util.function.Consumer;

import static java.lang.System.out;
import static java.util.Comparator.*;
import static java.util.stream.Collectors.*;

public class Exercise6 {

    public static void main(String[] args) {
        CountryDao countryDao = InMemoryWorldDao.getInstance();
        CityDao cityDao = InMemoryWorldDao.getInstance();
        Map<String, Optional<ContinentPopulatedCity>> continentsCapitals = countryDao.findAllCountries()
                .stream()
                .filter(country -> country.getCapital() > 0)
                .map(country -> new ContinentPopulatedCity(country.getContinent(), cityDao.findCityById(country.getCapital())))
                .collect(groupingBy(ContinentPopulatedCity::getKey, maxBy(comparing(cpc -> cpc.getValue().getPopulation()))));
        BiConsumer<String, Optional<ContinentPopulatedCity>> print= (k, v) -> {
            Consumer<ContinentPopulatedCity> continentPopulatedCityConsumer = cpc -> out.println(cpc.getKey() + ": " + v.get().getValue());
            v.ifPresent(continentPopulatedCityConsumer);
        };
        continentsCapitals.forEach(print);
    }

}

class ContinentPopulatedCity implements Map.Entry<String, City> {
    private String continent;
    private City city;

    public ContinentPopulatedCity(String continent, City city) {
        this.continent = continent;
        this.city = city;
    }

    @Override    public String getKey() {
        return continent;
    }

    @Override    public City getValue() {
        return city;
    }

    @Override    public City setValue(City city) {
        this.city = city;
        return city;
    }

}

The code given above will produce the following output:

South America: City [id=2890, name=Lima, population=6464693, countryCode=PER]
Asia: City [id=2331, name=Seoul, population=9981619, countryCode=KOR]
Europe: City [id=3580, name=Moscow, population=8389200, countryCode=RUS]
Africa: City [id=608, name=Cairo, population=6789479, countryCode=EGY]
North America: City [id=2515, name=Ciudad de M‚xico, population=8591309, countryCode=MEX]
Oceania: City [id=135, name=Canberra, population=322723, countryCode=AUS]

EXERCISE #7


Sort the countries by number of their cities in desending order:

package com.example.exercise;

import com.example.dao.CountryDao;
import com.example.dao.InMemoryWorldDao;
import com.example.domain.Country;

import java.util.Comparator;
import java.util.List;
import java.util.function.Predicate;

import static java.lang.String.format;
import static java.lang.System.out;
import static java.util.Comparator.comparing;
import static java.util.stream.Collectors.toList;

public class Exercise7 {

    public static void main(String[] args) {
        CountryDao countryDao = InMemoryWorldDao.getInstance();
        Comparator<Country> sortByNumOfCities = comparing(country -> country.getCities().size());
        Predicate<Country> countriesHavingNoCities = country -> country.getCities().isEmpty();
        List<Country> countries = countryDao.findAllCountries()
                .stream()
                .filter(countriesHavingNoCities.negate())
                .sorted(sortByNumOfCities.reversed())
                .collect(toList());
        countries.forEach(country -> out.println(format("%38s %3d", country.getName(), country.getCities().size())));
    }

}

The code given above will produce the following output:

                               China 363
                               India 341
                       United States 274
                              Brazil 250
                               Japan 248
                  Russian Federation 189
                              Mexico 173
                                       .
                                       .
                                       .
                            Barbados   1
                              Tuvalu   1
                                Niue   1
                Virgin Islands, U.S.   1
                              Brunei   1

EXERCISE #8

Find the list of movies having the genres "Drama" and "Comedy" only:

package com.example.exercise;

import com.example.domain.Genre;
import com.example.domain.Movie;
import com.example.service.InMemoryMovieService;
import com.example.service.MovieService;

import java.util.Collection;
import java.util.List;
import java.util.function.Predicate;

import static java.lang.String.format;
import static java.lang.System.out;
import static java.util.stream.Collectors.joining;
import static java.util.stream.Collectors.toList;

public class Exercise8 {

    public static void main(String[] args) {
        MovieService movieService = InMemoryMovieService.getInstance();
        Collection<Movie> movies = movieService.findAllMovies();
        Predicate<Movie> drama = movie -> movie.getGenres().stream().anyMatch(genre -> genre.getName().equals("Drama"));
        Predicate<Movie> comedy = movie -> movie.getGenres().stream().anyMatch(genre -> genre.getName().equals("Comedy"));
        Predicate<Movie> havingTwoGenresOnly = movie -> movie.getGenres().size() == 2;
        List<Movie> dramaAndComedyMovies = movies.stream()
                .filter(havingTwoGenresOnly.and(drama.and(comedy)))
                .collect(toList());
        dramaAndComedyMovies.forEach(movie -> out.println(format("%-32s: %12s", movie.getTitle(), movie.getGenres().stream().map(Genre::getName).collect(joining(",")))));
    }

}

The code given above will produce the following output:

Away We Go                      : Comedy,Drama
A Serious Man                   : Comedy,Drama
High Life                       : Comedy,Drama
Cold Souls                      : Comedy,Drama
Worlds Greatest Dad             : Comedy,Drama
My One and Only                 : Comedy,Drama
Sunshine Cleaning               : Comedy,Drama
The Vicious Kind                : Comedy,Drama
Defendor                        : Comedy,Drama
I Love You Phillip Morris       : Comedy,Drama

EXERCISE #9

Group the movies by the year and list them:

package com.example.exercise;

import com.example.domain.Movie;
import com.example.service.InMemoryMovieService;
import com.example.service.MovieService;

import java.util.Collection;
import java.util.Map;

import static java.lang.String.format;
import static java.lang.System.out;
import static java.util.Comparator.comparing;
import static java.util.Map.*;
import static java.util.stream.Collectors.*;

public class Exercise9 {

    public static void main(String[] args) {
        MovieService movieService = InMemoryMovieService.getInstance();
        Collection<Movie> movies = movieService.findAllMovies();
        Map<Integer, String> moviesByYear = movies.stream().collect(groupingBy(Movie::getYear, mapping(Movie::getTitle, joining(","))));
        moviesByYear.entrySet().stream().sorted(comparing(Entry::getKey)).forEach(entry -> out.println(format("%4d: %s", entry.getKey(), entry.getValue())));
    }

}

The code given above will produce the following output:

1940: The Return of Frank James
1944: Double Indemnity
1948: The Treasure of the Sierra Madre
1950: Sunset Blvd.
1951: A Streetcar Named Desire
1953: Stalag 17
1954: Them!,Shichinin no samurai,Dial M for Murder
1958: Vertigo
1960: Psycho
1963: The Great Escape
1968: The Party
1969: Butch Cassidy and the Sundance Kid,Easy Rider,The Wild Brunch
1973: Le magnifique
1975: Dog Day Afternoon
1976: Network,The Little Girl Who Lives Down the Lane
1977: Der amerikanische Freund,The Last Wave
1983: Rembetiko,Danton,The Outsiders,Scarface
1987: Empire of the Sun
1988: The Accused
1989: My Left Foot: The Story of Christy Brown
1991: My Own Private Idaho
1992: Of Mice and Men
1993: Tombstone,Germinal
1994: Before the Rain,Heavenly Creatures
1996: L appartement
1997: Bacheha-Ye aseman,The Rainmaker
2000: Almost Famous
2001: Yeopgijeogin geunyeo
2002: Ice Age
2003: Bom yeoreum gaeul gyeoul geurigo bom,Oldboy,Jeux Denfants,Keulraesik,Azumi
2004: Bin-jip,Nae meorisokui jiwoogae,Samaria,Wicker Park,2046,Voditel dlya Very
2005: Hiroshima,Hwal,Just Like Heaven,Azumi 2: Death or Love
2006: Deiji,Ice Age: The Meltdown,La Sconosciuta
2007: Paranormal Activity,Before the Devil Knows You are Dead,Broken English,Ex Drummer
2008: Adam Resurrected,Nothing But the Truth,100 Feet,Nordwand,The Other Man,God on Trial,Sunshine Cleaning,Oorlogswinter,Bin-mong,Pazar - Bir Ticaret Masalı,The Hurt Locker,My Only Sunshine,Karamazovi,Ghost Town,To Verdener,Sonbahar,Yip Man,Elegy,What Doesnt Kill You,Faubourg 36
2009: 500 Days Of Summer,Beyond a Reasonable Doubt,Gamer,Cheri,Dorian Gray,Inglourious Basterds,Invictus,Julie and Julia,Los abrazos rotos,Armored,Bornova Bornova,Coco avant Chanel,Nefes: Vatan sağolsun,Up,Whiteout,The Time Travelers Wife,Whatever Works,Anonyma - Eine Frau in Berlin,Zombieland,Weather Girl,Watchmen,Angels and Deamons,Away We Go,Last Ride,The Boys Are Back,The Tournament,A Serious Man,Saw VI,Ne te retourne pas,District 9,Extract,Five Minutes of Haven,High Life,The Proposal,Veronika Decides to Die,The Goods: Live Hard, Sell Hard,The Hangover,Public Enemies,Creation,Amelia,The Rebound,Powder Blue,The Men Who Stare at Goats,Bright Star,Case 39,Cold Souls,Moon,Worlds Greatest Dad,State of Play,The Brothers Bloom,My One and Only,Man Som Hatar Kvinnor,Mary and Max,The Limits of Control,A Perfect Getaway,My Sisters Keeper,Planet 51,I Love You, Man,Amelia,The Damned United,New York, I Love You,Fish Tank,The Informant!,The Courageous Heart of Irena Sendler,Storm,Triangle,2012,The Cry of the Owl,13B,El secreto de sus ojos,Surrogates,Kimssi pyoryugi,Uzak İhtimal,Daybreakers,Cairo Time,The Cove,Tenderness,Hachiko: A Dogs Story,The Box,Everybodys Fine,Peter and Vandy,Women in Trouble,Un prophete,The Vicious Kind,Bakjwi,Up in the air,Law Abiding Citizen,Nine,The Soloist,Agora,Motherhood,Neşeli Hayat,The Greatest,The Boondock Saints II: All Saints Day,The Private Lives of Pippa Lee,The Imaginarium of Doctor Parnassus,The Men Who Stare at Goats,Cloudy with a Chance of Meatballs,The Princess and the Frog,An Education,Avatar,Avatar 3D,Precious: Based on the Novel Push by Sapphire,The Blind Side,New Moon,Fantastic Mr. Fox,Sherlock Holmes,The Road,Man som hatar kvinnor,The Collector,Leaves of Grass,Brooklyns Finest,Alice,Duplicity,Harry Brown,Defendor,Brothers,Crazy Heart,Kıskanmak,Das weisse Band - Eine deutsche Kindergeschichte,The Lovely Bones,Eastern Plays,Cargo,Glorious 39,Fifty Dead Men Walking,Grey Gardens,Vavien,Lebanon,Harry Potter and the Half-Blood Prince,Slovenka,9:06,2081,The Electric Mist,Serious Moonlight,Ice Age: Dawn of the Dinosaurs,La doppia ora,A Single Man,Cracks,The Missing Person,Nowhere Boy,Chloe,Drag Me to Hell,Eloise's Lover,Başka Dilde Aşk,Air Doll
2010: From Paris with Love,Edge of Darkness,Shutter Island,The Bounty Hunter,Dear John,Extraordinary Measures,Leap Year,Yahşi Batı,I Love You Phillip Morris,You Dont Know Jack,Yip Man 2: Chung si chuen kei,Alice in Wonderland,Romantik Komedi,Veda,Sin Nombre,The Book of Eli,Unthinkable,Shrek Forever After


EXERCISE #10

Sort the countries by their population densities in descending order ignoring zero population countries:

package com.example.exercise;

import com.example.dao.InMemoryWorldDao;
import com.example.dao.WorldDao;
import com.example.domain.Country;

import java.util.Collection;
import java.util.Comparator;
import java.util.function.Predicate;

import static java.lang.System.out;
import static java.util.Comparator.comparingDouble;

public class Exercise10 {

    public static void main(String[] args) {
        WorldDao worldDao = InMemoryWorldDao.getInstance();
        Collection<Country> countries = worldDao.findAllCountries();
        Comparator<Country> populationDensityComparator = comparingDouble(country -> country.getPopulation() / country.getSurfaceArea());
        Predicate<Country> livesNobody = country -> country.getPopulation() == 0L;
        countries.stream().filter(livesNobody.negate()).sorted(populationDensityComparator.reversed())
                .forEach(out::println);
    }

}

The code given above will produce the following output:

Country [ name=Macao, population=473000]
Country [ name=Monaco, population=34000]
Country [ name=Hong Kong, population=6782000]
Country [ name=Singapore, population=3567000]
Country [ name=Gibraltar, population=25000]
.
.
.

Country [ name=Western Sahara, population=293000]
Country [ name=Pitcairn, population=50]
Country [ name=Falkland Islands, population=2000]
Country [ name=Svalbard and Jan Mayen, population=3200]
Country [ name=Greenland, population=56000]

You can download the source of the domain through this link.

Sunday, June 11, 2017

Javascript'de FilterMapReduce Kullanımı


Javascript, HTML 5 ile birlikte istemci tarafta, web tarayıcıda uygulama geliştirmek için standard bir dile dönüştü. HTML5 ile gelen WebSocket APILocal Storage APIBattery Status APIFullScreen APICanvas APIGeolocation APIVibration APIMultimedia APILocal Storage APIFile APIWebGL APIDrag-and-Drop API gibi bir yığın API'ye JavaScript'de yazdığımız uygulamadan erişebiliyoruz. Ancak Javascript kullanımı sadece ön uç (=front-end) ile  sınırlı değil. Nodejs ile arka uçta (=back-end) uygulama geliştirmek istediğinizde Javascript kullanmalısınız. JDK 8 ile birlikte gelen Java Sanal Makinası (=Java Virtual Machine), JavaScript ile yazılmış uygulamaları hızlı bir şekilde çalıştırabiliyor. Javascript'i hem ön uçta hem de arka uçta uygulama geliştirmek için kullanıyoruz.

Javascript'de verileri saklamak için dizi kullanıyoruz. Dizi özel bir tür nesne. Dizi, verileri anahtar-değer ikilileri olarak saklıyor. Dizide saklanan tüm veriler üzerinde işlem yapmak için döngü oluşturuyoruz. Döngüyü oluşturmanın çeşitli yöntemleri bulunuyor:

i. for ile döngüyü dizinin dışında oluşturuyoruz. İndis sıfırdan başlamak üzere dizinin son elemanının indis değerini alacak şekilde bir arttırılıyor:

var sum=0;
for (var i=0;i<countries.length;++i){
   var country = countries[i];
   sum += country.population;
}
console.log("Total population : "+ sum);

ii. for-each gösterimi ile döngüyü yine dizinin dışında oluşturuyoruz, ancak bu sefer dizi indisini düzenlemekle uğraşmıyoruz:

var sum=0;
for (var i in countries){
   var country = countries[i];
   sum += country.population;
}
console.log("Total population : "+ sum);

Dizide saklanan verileri işlemek için kullanılabilecek üçüncü ve en güçlü yöntem MapReduce yapısıdır. Şimdi örneklerle Javascript'de MapReduce çatısının kullanımını çalışalım. Önce üzerinde çalışacağımız veri modeline bir göz atalım. Verileri http://www.omegaegitim.com/resources/countries.json URL adresinden çekiyoruz:

let url = 'http://www.omegaegitim.com/resources/countries.json';
fetch( url)
.then( res => res.json() )
.then( ( countries ) => console.log( JSON.stringify(countries[215]) ) )
.catch( err => console.error( err ) );

Dizinin 215 indisli gözünde Türkiye ile ilgili bilgiler var:

{
  "capital": 3358,
  "cities": [
    {
      "district": "Istanbul",
      "_id": 3357,
      "name": "Istanbul",
      "population": 8787958
    },
    {
      "district": "Ankara",
      "_id": 3358,
      "name": "Ankara",
      "population": 3038159
    },
    {
      "district": "Izmir",
      "_id": 3359,
      "name": "Izmir",
      "population": 2130359
    },
    {
      "district": "Adana",
      "_id": 3360,
      "name": "Adana",
      "population": 1131198
    }, 
    . . . ,
    {
      "district": "Balikesir",
      "_id": 3418,
      "name": "Bandirma",
      "population": 90200
    }
  ],
  "_id": "TUR",
  "continent": "Asia",
  "gnp": 20000,
  "governmentForm": "Republic",
  "headOfState": "Ahmet Necdet Sezer",
  "indepYear": 1923,
  "lifeExpectancy": 85,
  "localName": "Türkiye",
  "name": "Turkey",
  "population": 66591000,
  "region": "Middle East",
  "surfaceArea": 774828
}

Listelenen bilgiler arasında ülkenin ISO 2 standardında kodu, ülkenin adı, yüzölçümü, başkenti, nüfusu, yönetim biçimi, yer aldığı kıta, Gayri Safi Milli Hasılası ve illeri gibi bilgiler yer alıyor. Bu veri modeline göre aşağıdaki soruları MapReduce çatısını kullanarak çözmeye çalışalım.

1. Kaç kıta olduğunu bulalım:

let url = 'http://www.omegaegitim.com/resources/countries.json';
var continents = [];
fetch(url)
.then(res => res.json())
.then((countries) => {
 continents= countries.map( country => country.continent ).reduce( 
  (continents,continent) => {
   if ( continents.indexOf(continent) < 0 ) {
    continents.push(continent);
   }
   return continents;
  }, [] );
  continents.sort();
  console.log( JSON.stringify(continents) );
})
.catch(err => console.error(err));

Yukarıdaki kodu çalıştırdığımızda 7 kıtanın listelendiğini görüyoruz:

["Africa","Antarctica","Asia","Europe","North America","Oceania","South America"]

2. Her bir kıtada kaç ülke olduğunu bulalım:

let url = 'http://www.omegaegitim.com/resources/countries.json';
fetch(url)
.then(res => res.json())
.then((countries) => {
 var result= countries.map( ( country ) => { return { name : country.name, continent: country.continent } } ).reduce( 
   (continents, o ) => {
    if (  !continents.hasOwnProperty(o.continent) ) {
     continents[o.continent] = { name: o.continent, countries: [ o.name ] };
    } else {
     continents[o.continent].countries.push( o.name );
    }
    return continents;
   }, {} );
 continents= [];
 for (var o in result) continents.push(result[o]);
 continents.sort( (o1,o2) => o1.name.localeCompare(o2.name) );
 continents.forEach( (continent) => console.log("There are "+continent.countries.length+" countries in "+continent.name) );
})
.catch(err => console.error(err));

Yukarıdaki kodu çalıştırdığımızda her bir kıtada kaç ülke olduğunun listesini görebiliyoruz:

There are 58 countries in Africa
There are 5 countries in Antarctica
There are 51 countries in Asia
There are 46 countries in Europe
There are 37 countries in North America
There are 28 countries in Oceania
There are 14 countries in South America

3. Şimdi her kıtanın GNP (Gross National Product)(=Gayri Safi Milli Hasıla) değerine göre en zengin ülkesini bulalım:

let url = 'http://www.omegaegitim.com/resources/countries.json';
fetch(url)
.then(res => res.json())
.then((countries) => {
 var result= countries.map( ( country ) => { return { name : country.name, continent: country.continent, gnp: country.gnp } } ).reduce( 
   (continents, o ) => {
    if (  !continents.hasOwnProperty(o.continent) ) {
     continents[o.continent] = { name: o.continent, richestCountry:  { name: o.name, gnp : o.gnp }};
    } else {
        var richestCountry= continents[o.continent].richestCountry;
     if ( richestCountry.gnp < o.gnp ){
      richestCountry.gnp= o.gnp;
      richestCountry.name= o.name;
     }     
    }
    return continents;
   }, {} );
 continents= [];
 for (var o in result) continents.push(result[o]);
 continents.sort( (o1,o2) => o1.name.localeCompare(o2.name) );
 continents.forEach( (continent) => console.log("Richest country in "+continent.name+" is " + continent.richestCountry.name + " with " + continent.richestCountry.gnp + " gnp." ) );
})
.catch(err => console.error(err));

Yukarıdaki kodu çalıştırdığımızda her bir kıtadaki en zengin ülkenin listesini görebiliyoruz:

Richest country in Africa is South Africa with 116729 gnp.
Richest country in Antarctica is Antarctica with 0 gnp.
Richest country in Asia is Japan with 3787042 gnp.
Richest country in Europe is Germany with 2133367 gnp.
Richest country in North America is United States with 8510700 gnp.
Richest country in Oceania is Australia with 351182 gnp.
Richest country in South America is Brazil with 776739 gnp.

4. En az bir kişinin yaşadığı dünya ülkelerinin minimum, maksimum ve ortalama nüfus sayılarını hesaplayalım:

let url = 'http://www.omegaegitim.com/resources/countries.json';
fetch(url)
.then(res => res.json())
.then((countries) => {
 var statistics= countries.filter( country => country.population > 0 )
                          .map( ( country ) => { return { name : country.name, population: country.population } } ).reduce( 
   ( statistics, o ) => {
    statistics.counter++;
    statistics.total += o.population;
    if ( isNaN(statistics.minimum)|| statistics.minimum > o.population)   
       statistics.minimum= o.population;
    if ( isNaN(statistics.maximum) || statistics.maximum < o.population)   
       statistics.maximum= o.population;
    statistics.average= statistics.total/statistics.counter;
    return statistics;
   }, { minimum: NaN, maximum: NaN, counter: 0, average: 0, total: 0} );
 console.log(JSON.stringify(statistics));
})
.catch(err => console.error(err));

İşte nüfus ile ilgili istatistikler:

{"minimum":50,"maximum":1277558000,"counter":232,"average":26201506.25,"total":6078749450}

5. Bir önceki hesaplamayı bu sefer kıta özelinde yapalım ve her bir kıta için en az bir kişinin yaşadığı kıta ülkelerinin minimum, maksimum ve ortalama nüfus sayılarını hesaplayalım:

let url = 'http://www.omegaegitim.com/resources/countries.json';
fetch(url)
.then(res => res.json())
.then((countries) => {
 var result= countries.filter( country => country.population > 0 )
                          .map( ( country ) => { return { name : country.name, population: country.population, continent: country.continent } } ).reduce( 
   ( statistics, o ) => {
       if (!statistics.hasOwnProperty(o.continent)){
       statistics[o.continent]= { continent: o.continent, minimum: NaN, maximum: NaN, counter: 0, average: 0, total: 0 };
    }
    var stat= statistics[o.continent];
    stat.counter++;
    stat.total += o.population;
    if ( isNaN(stat.minimum)|| stat.minimum > o.population)   
       stat.minimum= o.population;
    if ( isNaN(stat.maximum) || stat.maximum < o.population)   
       stat.maximum= o.population;
    stat.average= stat.total/stat.counter;
    return statistics;
   }, {} );
 statistics= [];
 for (var o in result) statistics.push(result[o]);
 statistics.sort( (o1,o2) => o1.continent.localeCompare(o2.continent) );
 statistics.forEach( stat => console.log(JSON.stringify(stat)) );
})
.catch(err => console.error(err));

İşte merak ettiğimiz sonuçlar:

{"continent":"Africa","minimum":6000,"maximum":111506000,"counter":57,"average":13762719.298245614,"total":784475000}
{"continent":"Asia","minimum":286000,"maximum":1277558000,"counter":51,"average":72647562.74509804,"total":3705025700}
{"continent":"Europe","minimum":1000,"maximum":146934000,"counter":46,"average":15871186.956521738,"total":730074600}
{"continent":"North America","minimum":7000,"maximum":278357000,"counter":37,"average":13053864.864864865,"total":482993000}
{"continent":"Oceania","minimum":50,"maximum":18886000,"counter":27,"average":1125968.5185185184,"total":30401150}
{"continent":"South America","minimum":2000,"maximum":170115000,"counter":14,"average":24698571.42857143,"total":345780000}

6. Şimdi 4'de yaptığımız analizi biraz değiştireceğiz. Minimum, maksimum ve ortalama nüfusu hesaplamıştık. Üzerinde en az bir kişinin yaşıdığı ülkeler arasında minimum ve maksimum nüfusa sahip ülkeleri bulalım:

let url = 'http://www.omegaegitim.com/resources/countries.json';
fetch(url)
.then(res => res.json())
.then((countries) => {
 var statistics= countries.filter( country => country.population > 0 ).reduce( 
   ( statistics, country ) => {
       if (statistics.lowestPopulatedCountry==null || statistics.lowestPopulatedCountry.population < country.population)
       statistics.lowestPopulatedCountry= country;
       if (statistics.highestPopulatedCountry==null || statistics.highestPopulatedCountry.population > country.population)
       statistics.highestPopulatedCountry= country;
       return statistics;
   }, { lowestPopulatedCountry: null, highestPopulatedCountry: null} );
 console.log("Lowest Populated Country is " 
     + statistics.lowestPopulatedCountry.name 
     + " with " 
     + statistics.lowestPopulatedCountry.population 
     + " population.");
 console.log("Highest Populated Country is " 
     + statistics.highestPopulatedCountry.name 
     + " with " 
     + statistics.highestPopulatedCountry.population 
     + " population.");
})
.catch(err => console.error(err));

İşte en az ve en çok kalabalık ülkeler:

Lowest Populated Country is China with 1277558000 population.
Highest Populated Country is Pitcairn with 50 population.

Sonuçlar şaşırtıcı olmadı ve en kalabalık ülke olarak Çin'i bulduk.

7. Bir önceki incelemeyi bu kez kıta özelinde yapalım ve kıtalardaki en az ve en fazla nüfusa sahip ülkeleri bulalım:

let url = 'http://www.omegaegitim.com/resources/countries.json';
fetch(url)
.then(res => res.json())
.then((countries) => {
 var result= countries.filter( country => country.population > 0 ).reduce( 
   ( statistics, country ) => {
       if (!statistics.hasOwnProperty(country.continent)){
       statistics[country.continent]= { continent: country.continent, lowestPopulatedCountry: country, highestPopulatedCountry: country };
    } else {
     if (statistics[country.continent].lowestPopulatedCountry.population < country.population)
      statistics[country.continent].lowestPopulatedCountry= country;
     if (statistics[country.continent].highestPopulatedCountry.population > country.population)
        statistics[country.continent].highestPopulatedCountry= country;
    }   
    return statistics;
   }, {} );
 statistics= [];
 for (var o in result) statistics.push(result[o]);
 statistics.sort( (o1,o2) => o1.continent.localeCompare(o2.continent) );
 statistics.forEach( stat => {
   console.log("Lowest Populated Country in " + stat.continent + " is " 
        + stat.lowestPopulatedCountry.name 
        + " with " 
        + stat.lowestPopulatedCountry.population 
        + " population.");
    console.log("Highest Populated Country in " + stat.continent + " is " 
        + stat.highestPopulatedCountry.name 
        + " with " 
        + stat.highestPopulatedCountry.population 
        + " population."); 
 } );
})
.catch(err => console.error(err));

İşte kıtalardaki en az ve en çok kalabalık ülkelerin listesi:

Lowest Populated Country in Africa is Nigeria with 111506000 population.
Highest Populated Country in Africa is Saint Helena with 6000 population.
Lowest Populated Country in Asia is China with 1277558000 population.
Highest Populated Country in Asia is Maldives with 286000 population.
Lowest Populated Country in Europe is Russian Federation with 146934000 population.
Highest Populated Country in Europe is Holy See (Vatican City State) with 1000 population.
Lowest Populated Country in North America is United States with 278357000 population.
Highest Populated Country in North America is Saint Pierre and Miquelon with 7000 population.
Lowest Populated Country in Oceania is Australia with 18886000 population.
Highest Populated Country in Oceania is Pitcairn with 50 population.
Lowest Populated Country in South America is Brazil with 170115000 population.
Highest Populated Country in South America is Falkland Islands with 2000 population.

8. Ülkelerin, en az nüfusa ve en çok nüfusa sahip şehirlerini bulalım:

let url = 'http://www.omegaegitim.com/resources/countries.json';
fetch(url)
.then(res => res.json())
.then((countries) => {
 var result= countries.filter( country => country.population > 0 ).reduce( 
   ( statistics, country ) => {
       statistics[country.name]= { name: country.name };
       cityStatistics = country.cities.reduce( 
        ( cityStatistics , city ) => {
      if (cityStatistics.lowestPopulatedCity==null || cityStatistics.lowestPopulatedCity.population < city.population)
       cityStatistics.lowestPopulatedCity= city;
      if (cityStatistics.highestPopulatedCity==null || cityStatistics.highestPopulatedCity.population > city.population)
       cityStatistics.highestPopulatedCity= city;         
      return cityStatistics;
     }     
    , { lowestPopulatedCity: null, highestPopulatedCity: null } ); 
                statistics[country.name].lowestPopulatedCity= cityStatistics.lowestPopulatedCity;
    statistics[country.name].highestPopulatedCity= cityStatistics.highestPopulatedCity;
    return statistics;
   }, {} );
 console.log(result)  ;
 statistics= [];
 for (var o in result) statistics.push(result[o]);
 console.log(statistics.length);
 statistics.sort( (o1,o2) => o1.name.localeCompare(o2.name) );
 statistics.forEach( stat => {
   console.log("Lowest Populated City in " + stat.name + " is " 
        + stat.lowestPopulatedCity.name 
        + " with " 
        + stat.lowestPopulatedCity.population 
        + " population.");
    console.log("Highest Populated City in " + stat.name + " is " 
        + stat.highestPopulatedCity.name 
        + " with " 
        + stat.highestPopulatedCity.population 
        + " population."); 
 } );
})
.catch(err => console.error(err));

İşte ülkelerin şehirlerinin durumu:

Lowest Populated City in Afghanistan is Herat with 186800 population.
Highest Populated City in Afghanistan is Mazar-e-Sharif with 127800 population.
Lowest Populated City in Albania is Tirana with 270000 population.
Highest Populated City in Albania is Tirana with 270000 population.
Lowest Populated City in Algeria is Alger with 2168000 population.
Highest Populated City in Algeria is Ghardaïa with 89415 population.
. . .
. . .
. . .
Lowest Populated City in Zambia is Lusaka with 1317000 population.
Highest Populated City in Zambia is Luanshya with 118100 population.
Lowest Populated City in Zimbabwe is Harare with 1410000 population.
Highest Populated City in Zimbabwe is Gweru with 128037 population.

Şu ana kadar MapReduce kullanarak yaptığımız hesaplamaları şimdi paralelleştirmeye çalışacağız. Bunun için Parallel JS kütüphanesini kullanacağız. Tüm ülkelerin toplam nüfusunu bu kez paralel çalışan MapReduce ile bulmaya çalışalım:

let url = 'countries.json';
  fetch(url)
  .then(res => res.json())
  .then((countries) => {
      var task = new Parallel(countries);
      var async= task.map( country => country.population )
                  .reduce( (population) => { return population[0]+population[1]; } , 0)
      async.then( (result) => console.log(result) );
  });