Sunday, April 30, 2017
Friday, April 28, 2017
Thursday, April 27, 2017
Wednesday, April 26, 2017
Tuesday, April 25, 2017
How to be an Online Session Musician: Part 1
A session musician is someone who’s paid to add their instrumentation to a recording or live event on a casual basis. In terms of recording you’d previously go to a studio, often in a major city.
Whilst this still happens with the advent of the internet, plus affordable recording equipment, session musicians can now work from home and with anyone anywhere in the world.
This has opened up the industry but increased competition, In simple terms, there’s more work to go around but more people than ever with whom to compete.
In this tutorial I’ll outline the practicalities you’ll need to consider to become an online session musician.
Qualification
This almost goes without saying, but let’s be clear: this isn’t a job for beginners.
You either need to be completely amazing at one style or facet of playing, or a musical chameleon, the sonic equivalent of a Swiss Army knife.
Unless you’re a specialist, someone who can build a reputation as the go-to person for a particular kind of playing, you’ll have to be prepared to turn your hand to whatever’s thrown at you.
That said…
Skill
It’s tempting to take the scattergun approach, and offer or agree to everything that’s thrown your way, especially in the early days. However, you run the risk of either declining a lot of enquiries, or worse, delivering sub-standard work because a job didn’t really fit your skill set. Neither will enhance your reputation.
It’s better to be very clear from the start as to what you’re offering. If you’re not sure how to describe your skills, take a look at existing ads from other session players. If nothing else, you may spot a gap in the market!
Hours
If you’re working with clients all over the world, you’ll be keeping some strange hours. If your home or working life is flexible enough to accommodate this, that’s fine.
If, however, your family already don’t see enough of you, doing this won’t improve matters.
If you need to keep strict hours make that clear to clients at the outset. You could even set up a Google calendar, or similar, so that potential clients can see when you’re available.
It’s Not All About You
When working on any project you’re likely to form an idea of what’s best in terms of what you can provide. Happily some clients will want to be guided by you, especially if they’ve no experience of your instrument or are new to arranging music.
Other clients, however, will have a strong opinion as to what they want, which may well differ from your own. You’ll need to be comfortable with playing what they’re asking for instead of what you want.
Respectfully offer your opinion, by all means, as the client might not have considered what you’re proposing. If they’re adamant, play what’s required of you.
Always remember: the man who pays the piper calls the tune.
Speaking of money…
Earning a Living
If you’re exceptionally talented, hard-working, network like crazy and get some lucky breaks, you might find yourself earning comfortably, thanks to some high-profile bookings.
This is exceptional.
Assuming that you’re not a high-profile professional, you can typically earn a few hundred a month.
Supply and Demand
It’s also worth knowing that, like a lot of jobs in the music world, there are good times and bad. Demand can be somewhat seasonal. For example, you might be rushed off your feet in the run-up to Christmas but find January extremely quiet.
Bearing all of this in mind it’s better to view this kind of work as an additional revenue stream rather than your sole income.
Declaration
When it comes to income, don’t forget to declare any such earnings on a tax return, even if you think you’re not earning much. If you’re new to this, go online, find the local tax office and contact them for advice.
Everybody Wants Some
A good way of getting work and building a reputation is through a web-based agency or marketplace. This allows you to set-up a shop front to advertise your services as well as bringing you enquiries. A typical example, and one that I use, is fiverr.com
If you go down this route, however, you need to factor a few things into your costs.
Commission
Such sites are going to want paying for helping you find work and fees are typically 10-20% per transaction.
Exchange Rates
Unsurprisingly, some of the biggest sites are US-based and work in dollars. As exchange rates will fluctuate, earnings won’t be fixed or guaranteed.
More Fees
Some sites will pay you via companies such as PayPal, so you’ll need to be registered accordingly. They in turn will take their own commission for handling your money.
Factoring this in, plus commission fees, means that you’ll receive around 60% of what the buyer originally paid. You'll need to set your prices to offset at least some of this whilst still remaining competitive.
Waiting to Get Paid
As with a lot of financial transactions, buyers are allowed a cooling off period after a purchase, in case they change their minds. This means that you’ll probably have to wait 7-14 days before payment is released to you.
Conclusion
Online session work can become a way of earning some extra money, provided you have the time and the skills to pursue it. In summation, bear in mind the following:
- Be good at what you do
- Be clear as to what you’re offering
- Regular hours can be a struggle
- Work isn’t guaranteed
- Play what’s required
- Declare earnings, whatever they are
- Factor fees into your prices
In the next tutorial, I’ll show you what you’ll need to get started, such as equipment.
Getting Started With Chart.js: Scales
In the last four tutorials, you have learned a great deal about Chart.js. After reading the first four tutorials, you should now be able to customize the tooltips and labels, change the fonts, and create different chart types. One aspect of Chart.js that has not been yet covered in this series is scales.
Scales have changed a lot since version v1.0 of the library and are now much more powerful. In this tutorial, you will learn how to manipulate scales and control their appearance using different options that the library provides.
Changing Grid Lines
All the configuration options for grid lines are nested under the scale option in the gridLines key. This key defines options to customize the grid lines that run perpendicular to the axes. Let's change the grid lines of the line chart that you created in the line and bar charts tutorial.
You can show or hide the grid lines of a chart by using the display key. Its initial value is true, so the grid lines are shown by default. The color of the grid lines can be specified by using the color key. If you want the grid lines to be composed of dashes instead of being solid lines, you can provide a value for the length and spacing of dashes as a value of the borderDash key. You can set the width and color of the lines for the first or zero index using the zeroLineWidth and zeroLineColor keys respectively.
In all the charts up to this point, the scales did not have any descriptive text to let the viewers know what a particular axis represented. This can make the charts less useful. For example, if you see a plot of car speed and can't figure out the unit in which the speed is plotted on the y-axis, the chart is pretty much useless.
You can show or hide the scale labels on a chart by using the display key. The scale labels are hidden by default. The text that should be shown on these scales can be specified using the labelString key. You can also control the font color, family, size, and style using the fontColor, fontFamily, fontSize, and fontStyle keys respectively.
Here is the same old car speed chart with a different set of chart options specified.
var chartOptions = {
legend: {
display: true,
position: 'top',
labels: {
boxWidth: 80,
fontColor: 'black'
}
},
scales: {
xAxes: [{
gridLines: {
display: false,
color: "black"
},
scaleLabel: {
display: true,
labelString: "Time in Seconds",
fontColor: "red"
}
}],
yAxes: [{
gridLines: {
color: "black",
borderDash: [2, 5],
},
scaleLabel: {
display: true,
labelString: "Speed in Miles per Hour",
fontColor: "green"
}
}]
}
};
There is one more key called offsetGridLines. When set to true, it shifts the labels to the middle of the grid lines. This is generally useful when creating bar charts.
Configuring Linear Scales
Linear scales are used to chart numerical data. These scales can be created on either the x or y axis. In most cases, Chart.js automatically detects the minimum and maximum values for the scales. However, this can result in some confusion.
Let's say you want to plot the marks of students in a class. If the maximum marks for the test were 200 but none of the students scored more than 180 marks, the scale will most probably max out at 180. In such cases, the readers will have no way of knowing if the maximum marks were 180 or 200.
Chart.js has multiple built-in options that let you control different keys for scales. You can specify the minimum and maximum value for scales using the min and max keys. The step size of the scales can be controlled using the stepSize property. This way, you can determine how many grid lines should be drawn on the chart. Another way to set a limit on the number of grid lines and ticks shown on a chart is to use the maxTicksLimit key.
Here is the code to specify the minimum and maximum scale values for the horizontal scale on the bar chart for a previous tutorial of this series.
var chartOptions = {
scales: {
yAxes: [{
barPercentage: 0.5,
gridLines: {
display: false
}
}],
xAxes: [{
gridLines: {
zeroLineColor: "black",
zeroLineWidth: 2
},
ticks: {
min: 0,
max: 6500,
stepSize: 1300
},
scaleLabel: {
display: true,
labelString: "Density in kg/m3"
}
}]
},
elements: {
rectangle: {
borderSkipped: 'left',
}
}
};
Just like the linear scale, you can also create logarithmic scales to plot values on your chart. In this case, logarithmic interpolation is used to determine the position of a point on the axes. These scales can also be placed on both the x-axis and the y-axis.
Configuring Radial Linear Scales
This scale type is used for radar and polar area chart types. Unlike the regular linear scale, this one overlays the chart area instead of being positioned on the axis.
There are a lot of keys specifically targeted at radial scales. For example, you can use the lineArc key to specify if the grid lines should be circular or straight. The default value is false for radar charts and true for polar area charts.
You can control the appearance of lines that radiate from the center of the chart to its point labels using the angleLines key. It accepts an object as its value. The object can contain keys to control the color and width of angle lines. You can hide the angle lines by setting the value of display key to false. The color and width of angle lines can be controlled using the color and lineWidth keys.
The appearance of point labels can be controlled using the pointLabels key. Just like angle lines, this key also accepts an object as its value. Please note that these options only have an effect when the value of lineArc is set to false. The font color, family, size, and style can be specified using the fontColor, fontFamily, fontSize, and fontStyle keys.
You can also set a minimum and maximum value for the scale using the min and max keys. The step size and maximum number of ticks on the scale can be specified using the stepSize and maxTicksLimit keys. These options are available under the ticks key. Some other keys available inside ticks are showLabelBackdrop, backdropColor, backdropPaddingX, and backDropPaddingY. You can use them to show or hide the background behind the tick labels and control its width, height, and color.
You can also use the gridLines key that we used for the line chart to set a color and width for the grid lines in a radar chart. Here are some options to create a radar chart with customized scales.
var chartOptions = {
scale: {
gridLines: {
color: "black",
lineWidth: 3
},
angleLines: {
display: false
},
ticks: {
beginAtZero: true,
min: 0,
max: 100,
stepSize: 20
},
pointLabels: {
fontSize: 18,
fontColor: "green"
}
},
legend: {
position: 'left'
}
};
Configuring Time Scales
You can use a time scale to display times and dates on a chart. As mentioned in the introductory Chart.js tutorial, you need to include Moment.js in your projects to display time. One restriction on using a time scale is that it can be displayed only on the x-axis. All the configuration options for the time scale are located under the time sub-option.
To create a time scale, you have to set the value of the type key to time under the xAxes sub-option. After that, you can set the value of different keys under time. The unit which should be used to draw the ticks is set using the unit key.
The step of the unit can be specified using the unitStepSize key. The format in which the labels for the ticks should be displayed is specified using the displayFormats sub-option. You can read more about allowed string formats on the Moment.js website.
The format in which the time is displayed in the tooltips can be specified using the tooltipFormat key.
You can also round the time before plotting it on the chart using the round key. Be careful when setting a value for round. Let's say you set its value to hour and there are two times that need to be plotted on the chart. If one time is 5:30 AM and another is 5:50 AM, they will both be plotted on the 5:00 AM tick mark. Setting its value to minute will plot them on the 5:30 and 5:50 marks respectively.
The line chart plotted in the beginning of the tutorial could be plotted using a time scale with the help of the following code.
var chartOptions = {
legend: {
display: true,
position: 'top',
labels: {
boxWidth: 80,
fontColor: 'black'
}
},
scales: {
xAxes: [{
type: "time",
time: {
unit: 'hour',
unitStepSize: 0.5,
round: 'hour',
tooltipFormat: "h:mm:ss a",
displayFormats: {
hour: 'MMM D, h:mm A'
}
}
}],
yAxes: [{
gridLines: {
color: "black",
borderDash: [2, 5],
},
scaleLabel: {
display: true,
labelString: "Speed in Miles per Hour",
fontColor: "green"
}
}]
}
};
Final Thoughts
This tutorial helped you learn about different kinds of scales in Chart.js. You can now easily customize the scales in a chart by controlling their color, minimum and maximum value, number of ticks, and other such factors.
After reading all five tutorials in this series, you should now be able to create all kinds of charts available in Chart.js. I hope you liked the tutorial and the series.
If you’re looking for additional resources to study or to use in your work, check out what we have available in the Envato marketplace. If you have any questions, please let me know in the comments. Have you ever used this library in your own projects? Please share some tips in the comments.
Monday, April 24, 2017
Saturday, April 22, 2017
Friday, April 21, 2017
How to Create a Laravel Helper
To begin with, I would rather let the Laravel official site speak about helpers.
Laravel includes a variety of global "helper" PHP functions. Many of these functions are used by the framework itself; however, you are free to use them in your own applications if you find them convenient.
So, basically, helpers in Laravel are built-in utility functions that you can call from anywhere within your application. If they hadn't been provided by the core framework, you might have ended up developing your own helper classes.
Although the core provides a variety of helpers already, there’s always a chance that you’ll need your own and would like to develop one so you don’t have to repeat the same code here and there, thus enforcing better maintainability. You'll learn how to create a custom Laravel helper in this tutorial.
Helpers in Laravel
As we discussed earlier, there are plenty of helpers available in the core of the Laravel framework. They are grouped together based on the functionality they provide. Here’s a list of helper groups.
Arrays
Helpers in this group provide functionality to manipulate array elements. More often than not, you're in the situation where you want to perform different operations on array elements. So this is the place where you should look first to see if what you're looking for already exists.
Paths
I find helpers in this category most useful. They return the fully qualified path of different directories like app, storage, config, and the like. I bet you're using most of these helpers already in your Laravel application.
Strings
String manipulation is something inevitable in your day-to-day application development. Although there are plenty of string manipulation functions provided by PHP itself, you'll find a few more useful goodies in this section.
URLs
You'll find very few in this category, but they are used throughout the application. They are used to generate route, asset, and form action URLs.
Miscellaneous
This category contains helpers that provide a variety of functionalities ranging from logging to debugging and many more.
For a complete reference of Laravel helpers, there's no better place than the official documentation.
Create Your First Custom Helper
Now you have a basic understanding of Laravel helpers and what they are used for. In this section, I’ll go ahead and demonstrate how you can create your own custom helper that can be used globally in your Laravel application.
To keep things simple and easy to understand, it’ll be a pretty basic helper that takes a userid and returns a username as a response. Of course, that doesn’t sound fancy, but I believe it’s enough to demonstrate the concept, and you could always extend it to fulfill your complex requirements.
I assume that you have a users table in your database and it has at least two fields—userid and username.
The Skeleton of a Laravel Helper
Before we move ahead and actually create the files, let’s have a quick look at the files that we’re going to create in the rest of the article.
app/Helpers/Envato/User.php: It’s our helper file that holds the logic of our helper.app/Providers/EnvatoServiceProvider.php: It’s a custom service provider file that loads our custom helper file.config/app.php: In this file, we’ll declare our custom service provider, and it also helps us to define an alias to our helper so that we don’t have to use the fully qualified class name of our helper.routes/web.php: A pretty standard Laravel route file where we’ll actually test our helper.
Create Helper Files
Although you could place your helper files anywhere in your application, the more intuitive and standard way suggests that it should go under your app directory.
So go ahead and create a Helpers/Envato directory under app and create a User.php file with the following contents. Of course, you could place it directly under the app or app/Helpers directory, but providing that extra level allows us to organize our helpers in good shape, specifically when you’re going to have plenty of them.
<?php
//app/Helpers/Envato/User.php
namespace App\Helpers\Envato;
use Illuminate\Support\Facades\DB;
class User {
/**
* @param int $user_id User-id
*
* @return string
*/
public static function get_username($user_id) {
$user = DB::table('users')->where('userid', $user_id)->first();
return (isset($user->username) ? $user->username : '');
}
}
The file starts with a pretty standard namespace declaration:
namespace App\Helpers\Envato;
The purpose of our custom helper is to retrieve a username based on a userid. Hence, we need to interact with the database, and that forces us to include DB Facade.
use Illuminate\Support\Facades\DB;
For those who are not familiar with Laravel Facades, it’s just another convenient way to access the objects in service containers. Alternatively, you could have used dependency injection.
Moving ahead, there comes the concrete implementation of our helper. As you can see, there’s a static method that defines the logic of retrieving a username based on a userid.
$user = DB::table('users')->where('userid', $user_id)->first();
The $user object holds the database record with the matching userid. Finally, the method returns the username as a response in the following statement.
return (isset($user->username) ? $user->username : '');
That’s it as far as our helper file is concerned.
Now we’ve created our helper file, but the question is how are you going to use it? Two quick solutions come to my mind:
- You can include our helper filename in the
composer.json, so that it gets auto-loaded. Then, you could straight away call the static method of our helper class. - You can go ahead and create a Laravel service provider that allows you to register your custom helper file so that the Laravel framework loads it along with other dependencies. Register that service provider in the Laravel configuration and create an alias to use your helper.
Of course, the first one is pretty quick and easy to implement, and you might be tempted to do so, but I would rather suggest the latter one as it looks like more of an artisan way and is more maintainable.
Move to the command line and run the following command in your application root to create a new service provider.
$php artisan make:provider EnvatoServiceProvider Provider created successfully.
You should see the message that confirms that it’s created successfully under the app/Providers directory.
Open that file and you should already see two methods out there. The important one in the context of this article is the register method. Yes, it’s empty at the moment, so let’s feed in some stuff to make it more useful.
public function register()
{
require_once app_path() . '/Helpers/Envato/User.php';
}
The register method is used to register your dependencies, and we’ve exactly done that. We’ve included our custom helper file.
Here’s how the app/Providers/EnvatoServiceProvider.php file should look after modifications.
<?php
// app/Providers/EnvatoServiceProvider.php
namespace App\Providers;
use Illuminate\Support\ServiceProvider;
class EnvatoServiceProvider extends ServiceProvider
{
/**
* Bootstrap the application services.
*
* @return void
*/
public function boot()
{
//
}
/**
* Register the application services.
*
* @return void
*/
public function register()
{
require_once app_path() . '/Helpers/Envato/User.php';
}
}
So it’s all pretty good so far. We have our custom helper and service provider on the table.
Next, we need to inform Laravel about our service provider so that it can load it during bootstrapping. Let’s open the config/app.php and add the following entry in the providers array at the end.
App\Providers\EnvatoServiceProvider::class,
To use our helper in a convenient way, we could create an alias as well. So let’s do that by adding the following entry in the aliases array at the end in the same file.
'EnvatoUser' => App\Helpers\Envato\User::class,
By defining this entry, we can call our helper by using the EnvatoUser keyword. Pretty convenient, huh? For your reference, here’s the complete config/app.php file.
<?php
// config/app.php
return [
/*
|--------------------------------------------------------------------------
| Application Name
|--------------------------------------------------------------------------
|
| This value is the name of your application. This value is used when the
| framework needs to place the application's name in a notification or
| any other location as required by the application or its packages.
*/
'name' => 'Laravel',
/*
|--------------------------------------------------------------------------
| Application Environment
|--------------------------------------------------------------------------
|
| This value determines the "environment" your application is currently
| running in. This may determine how you prefer to configure various
| services your application utilizes. Set this in your ".env" file.
|
*/
'env' => env('APP_ENV', 'production'),
/*
|--------------------------------------------------------------------------
| Application Debug Mode
|--------------------------------------------------------------------------
|
| When your application is in debug mode, detailed error messages with
| stack traces will be shown on every error that occurs within your
| application. If disabled, a simple generic error page is shown.
|
*/
'debug' => env('APP_DEBUG', false),
/*
|--------------------------------------------------------------------------
| Application URL
|--------------------------------------------------------------------------
|
| This URL is used by the console to properly generate URLs when using
| the Artisan command line tool. You should set this to the root of
| your application so that it is used when running Artisan tasks.
|
*/
'url' => env('APP_URL', 'http://localhost'),
/*
|--------------------------------------------------------------------------
| Application Timezone
|--------------------------------------------------------------------------
|
| Here you may specify the default timezone for your application, which
| will be used by the PHP date and date-time functions. We have gone
| ahead and set this to a sensible default for you out of the box.
|
*/
'timezone' => 'UTC',
/*
|--------------------------------------------------------------------------
| Application Locale Configuration
|--------------------------------------------------------------------------
|
| The application locale determines the default locale that will be used
| by the translation service provider. You are free to set this value
| to any of the locales which will be supported by the application.
|
*/
'locale' => 'en',
/*
|--------------------------------------------------------------------------
| Application Fallback Locale
|--------------------------------------------------------------------------
|
| The fallback locale determines the locale to use when the current one
| is not available. You may change the value to correspond to any of
| the language folders that are provided through your application.
|
*/
'fallback_locale' => 'en',
/*
|--------------------------------------------------------------------------
| Encryption Key
|--------------------------------------------------------------------------
|
| This key is used by the Illuminate encrypter service and should be set
| to a random, 32 character string, otherwise these encrypted strings
| will not be safe. Please do this before deploying an application!
|
*/
'key' => env('APP_KEY'),
'cipher' => 'AES-256-CBC',
/*
|--------------------------------------------------------------------------
| Logging Configuration
|--------------------------------------------------------------------------
|
| Here you may configure the log settings for your application. Out of
| the box, Laravel uses the Monolog PHP logging library. This gives
| you a variety of powerful log handlers / formatters to utilize.
|
| Available Settings: "single", "daily", "syslog", "errorlog"
|
*/
'log' => env('APP_LOG', 'single'),
'log_level' => env('APP_LOG_LEVEL', 'debug'),
/*
|--------------------------------------------------------------------------
| Autoloaded Service Providers
|--------------------------------------------------------------------------
|
| The service providers listed here will be automatically loaded on the
| request to your application. Feel free to add your own services to
| this array to grant expanded functionality to your applications.
|
*/
'providers' => [
/*
* Laravel Framework Service Providers...
*/
Illuminate\Auth\AuthServiceProvider::class,
Illuminate\Broadcasting\BroadcastServiceProvider::class,
Illuminate\Bus\BusServiceProvider::class,
Illuminate\Cache\CacheServiceProvider::class,
Illuminate\Foundation\Providers\ConsoleSupportServiceProvider::class,
Illuminate\Cookie\CookieServiceProvider::class,
Illuminate\Database\DatabaseServiceProvider::class,
Illuminate\Encryption\EncryptionServiceProvider::class,
Illuminate\Filesystem\FilesystemServiceProvider::class,
Illuminate\Foundation\Providers\FoundationServiceProvider::class,
Illuminate\Hashing\HashServiceProvider::class,
Illuminate\Mail\MailServiceProvider::class,
Illuminate\Notifications\NotificationServiceProvider::class,
Illuminate\Pagination\PaginationServiceProvider::class,
Illuminate\Pipeline\PipelineServiceProvider::class,
Illuminate\Queue\QueueServiceProvider::class,
Illuminate\Redis\RedisServiceProvider::class,
Illuminate\Auth\Passwords\PasswordResetServiceProvider::class,
Illuminate\Session\SessionServiceProvider::class,
Illuminate\Translation\TranslationServiceProvider::class,
Illuminate\Validation\ValidationServiceProvider::class,
Illuminate\View\ViewServiceProvider::class,
/*
* Package Service Providers...
*/
Laravel\Tinker\TinkerServiceProvider::class,
/*
* Application Service Providers...
*/
App\Providers\AppServiceProvider::class,
App\Providers\AuthServiceProvider::class,
// App\Providers\BroadcastServiceProvider::class,
App\Providers\EventServiceProvider::class,
App\Providers\RouteServiceProvider::class,
App\Providers\EnvatoServiceProvider::class,
],
/*
|--------------------------------------------------------------------------
| Class Aliases
|--------------------------------------------------------------------------
|
| This array of class aliases will be registered when this application
| is started. However, feel free to register as many as you wish as
| the aliases are "lazy" loaded so they don't hinder performance.
|
*/
'aliases' => [
'App' => Illuminate\Support\Facades\App::class,
'Artisan' => Illuminate\Support\Facades\Artisan::class,
'Auth' => Illuminate\Support\Facades\Auth::class,
'Blade' => Illuminate\Support\Facades\Blade::class,
'Broadcast' => Illuminate\Support\Facades\Broadcast::class,
'Bus' => Illuminate\Support\Facades\Bus::class,
'Cache' => Illuminate\Support\Facades\Cache::class,
'Config' => Illuminate\Support\Facades\Config::class,
'Cookie' => Illuminate\Support\Facades\Cookie::class,
'Crypt' => Illuminate\Support\Facades\Crypt::class,
'DB' => Illuminate\Support\Facades\DB::class,
'Eloquent' => Illuminate\Database\Eloquent\Model::class,
'Event' => Illuminate\Support\Facades\Event::class,
'File' => Illuminate\Support\Facades\File::class,
'Gate' => Illuminate\Support\Facades\Gate::class,
'Hash' => Illuminate\Support\Facades\Hash::class,
'Lang' => Illuminate\Support\Facades\Lang::class,
'Log' => Illuminate\Support\Facades\Log::class,
'Mail' => Illuminate\Support\Facades\Mail::class,
'Notification' => Illuminate\Support\Facades\Notification::class,
'Password' => Illuminate\Support\Facades\Password::class,
'Queue' => Illuminate\Support\Facades\Queue::class,
'Redirect' => Illuminate\Support\Facades\Redirect::class,
'Redis' => Illuminate\Support\Facades\Redis::class,
'Request' => Illuminate\Support\Facades\Request::class,
'Response' => Illuminate\Support\Facades\Response::class,
'Route' => Illuminate\Support\Facades\Route::class,
'Schema' => Illuminate\Support\Facades\Schema::class,
'Session' => Illuminate\Support\Facades\Session::class,
'Storage' => Illuminate\Support\Facades\Storage::class,
'URL' => Illuminate\Support\Facades\URL::class,
'Validator' => Illuminate\Support\Facades\Validator::class,
'View' => Illuminate\Support\Facades\View::class,
'EnvatoUser' => App\Helpers\Envato\User::class,
],
];
We’re almost there! We’ve done all the hard work to get here, and now we can reap the benefits of our custom helper.
Your First Custom Helper in Action
Again, to keep things simple and straightforward, we’ll define a basic Laravel route and call our helper from there!
Go ahead and create a routes/web.php file with the following contents.
<?php
// routes/web.php
/*
|--------------------------------------------------------------------------
| Web Routes
|--------------------------------------------------------------------------
|
| Here is where you can register web routes for your application. These
| routes are loaded by the RouteServiceProvider within a group which
| contains the "web" middleware group. Now create something great!
|
*/
Route::get('/envato-user-helper-demo', function () {
return EnvatoUser::get_username(1);
});
Does that need any explanation at all? We’ve just called the custom helper by the shorthand EnvatoUser::get_username, and it should return the username.
Of course, you can call our helper from anywhere in the application, be it a controller or view.
So that ends our story for today.
Conclusion
Helpers in Laravel are really a powerful feature, and I'm sure that as a developer you would love to extend that. And that was the topic of today—we went through the basics of the Laravel helper file structure and created a useful custom helper.
I hope you’ve enjoyed the article and it helps you create your own custom helpers in day-to-day Laravel application development.
Don’t hesitate to leave your comments and queries in the feed below. I also catch comments on my Twitter and respond to them as soon as I can!
Getting Started With Chart.js: Pie, Doughnut, and Bubble Charts
You have learned about four different chart types in Chart.js up to this point. The second tutorial of the series covered line and bar charts. The third tutorial discussed radar and polar area charts. In this tutorial, you will learn how to use Chart.js to create pie, doughnut, and bubble charts.
Creating Pie and Doughnut Charts
Pie and doughnut charts are useful when you want to show the proportion in which something is divided among different entities. For example, you can use pie charts to show the percentage of males, females, and young ones of lions in a wildlife park, or the percentage of votes that different candidates got in an election.
Pie charts are only helpful when you want to compare one specific parameter or set of data. An important thing to keep in mind is that you cannot use pie charts to plot entities whose values are zero because the angle of the sectors in a pie chart depends on the magnitude of the data points.
This means that any entity whose share is zero won't be shown on the chart at all. Similarly, you cannot plot negative values on a pie chart. You can create pie charts in Chart.js by setting the type key to pie. Similarly, you can create doughnut charts by setting the type key to doughnut. Here is an example of creating these two charts:
var pieChart = new Chart(votesCanvas, {
type: 'pie',
data: votesData,
options: chartOptions
});
var doughnutChart = new Chart(votesCanvas, {
type: 'doughnut',
data: votesData,
options: chartOptions
});
Let's create a pie chart which shows the oil exports data of the top five countries in 2015. The data is in US billion dollars.
var data = {
labels: [
"Saudi Arabia",
"Russia",
"Iraq",
"United Arab Emirates",
"Canada"
],
datasets: [
{
data: [133.3, 86.2, 52.2, 51.2, 50.2],
backgroundColor: [
"#FF6384",
"#63FF84",
"#84FF63",
"#8463FF",
"#6384FF"
]
}]
};
You can control the appearance of the above chart using different keys like cutoutPercentage, which defines the percentage of the chart that is cut out of the middle. You can also specify the start angle of the chart using the rotation key. Similarly, you can also specify the angle that the chart sweeps while plotting the data using the circumference key.
There are two different animations that can be activated while drawing a chart. You can specify if the chart should have a rotation animation using the animateRotate key. Similarly, you can also specify if the doughnut chart should be scaled from the center using the animateScale key. The value of animateRotate is set to true by default, and the value for animateScale is set to false by default.
As usual, you can control the background color, border color, and border width of all the data points using the backgroundColor, borderColor, and borderWidth keys respectively. Similarly, the hover values of all these properties can be controlled using the hoverBackgroundColor, hoverBorderColor, and hoverBorderWidth keys.
Here is the code to create a doughnut chart for the above data. Setting the value for rotation equal to -Math.PI takes that starting point 180 degrees anti-clockwise. Then, setting the value of circumference to Math.PI makes the chart only span a semicircle.
var oilData = {
labels: [
"Saudi Arabia",
"Russia",
"Iraq",
"United Arab Emirates",
"Canada"
],
datasets: [
{
data: [133.3, 86.2, 52.2, 51.2, 50.2],
backgroundColor: [
"#FF6384",
"#63FF84",
"#84FF63",
"#8463FF",
"#6384FF"
],
borderColor: "black",
borderWidth: 2
}]
};
var chartOptions = {
rotation: -Math.PI,
cutoutPercentage: 30,
circumference: Math.PI,
legend: {
position: 'left'
},
animation: {
animateRotate: false,
animateScale: true
}
};
Creating Bubble Charts
Bubble charts are used to plot or display three dimensions (p1, p2, p3) of data on a chart. The position and size of the bubbles determines the value of these three data points. The horizontal axis represents the first data point (p1), the vertical axis represents the second data point (p2), and the area of the bubble is used to represent the value of the third data point (p3).
One thing that you should keep in mind is that the magnitude of the third data point is not represented by the radius of the bubbles but their area. Now, the area of a circle is proportional to the square of the radius. This means that you have to make sure that the radius of the bubble that you plot is proportional to the square root of the magnitude of the third data point.
You can create bubble charts in Chart.js by setting the value of the type key to bubble. Here is an example of creating a bubble chart.
var bubbleChart = new Chart(popCanvas, {
type: 'bubble',
data: popData,
options: chartOptions
});
Let's plot the weight of different items kept in a room using a bubble chart. The data for the chart needs to be passed in the form of an object. The data object needs to have the following interface for it to be plotted properly.
{
x: <Number>,
y: <Number>,
r: <Number>
}
One important difference between bubble charts and all other charts is that the bubble radius is not scaled with the chart.
For example, the width of bars in a bar chart can increase or decrease based on the chart size. The same thing does not happen with bubble charts. The radius of the bubbles is always equal to the exact number of pixels that you specified. It makes sense because in this chart type, the radius is actually being used to represent real data.
Let's create a bubble chart to plot the population of deer at different spots in a forest.
var popData = {
datasets: [{
label: ['Deer Population'],
data: [{
x: 100,
y: 0,
r: 10
}, {
x: 60,
y: 30,
r: 20
}, {
x: 40,
y: 60,
r: 25
}, {
x: 80,
y: 80,
r: 50
}, {
x: 20,
y: 30,
r: 25
}, {
x: 0,
y: 100,
r: 5
}],
backgroundColor: "#FF9966"
}]
};
Since the radius here is proportional to the square root of the actual magnitude, the number of deer at (80, 80) is 100 times more than the number of deer at (0, 100).
Just like all other chart types, you can control the background color, border color, and border width of plotted points using the backgroundColor, borderColor, and borderWidth keys. Similarly, you can also specify the hover background color, hover border color, and hover border width using the hoverBackgroundColor, hoverBorderColor, and hoverBorderWidth keys.
You can also specify the additional radius that you want to add to different bubbles on hover using the hoverRadius key. Remember that this radius is added to the original value to draw the hovered bubble. If the original bubble had a radius of 10 and hoverRadius is set to 5, the radius of the bubble on hover will be equal to 15.
var popData = {
datasets: [{
label: ['Deer Population'],
data: [{
x: 100,
y: 0,
r: 10
}, {
x: 60,
y: 30,
r: 20
}, {
x: 40,
y: 60,
r: 25
}, {
x: 80,
y: 80,
r: 50
}, {
x: 20,
y: 30,
r: 25
}, {
x: 0,
y: 100,
r: 5
}],
backgroundColor: "#9966FF",
hoverBackgroundColor: "#000000",
hoverBorderColor: "#9966FF",
hoverBorderWidth: 5,
hoverRadius: 5
}]
};
The above parameters will create the following bubble chart.
Final Thoughts
In this tutorial, you learned about three more chart types available in Chart.js. You should now be able to choose the appropriate chart type to plot your data and set specific values for different keys to control their appearance. In the next tutorial, you will learn how to manipulate the scales for different chart types.
I hope you liked this tutorial. If you have any questions, please let me know in the comments.
Thursday, April 20, 2017
Introduction to Parallel and Concurrent Programming in Python
Python is one of the most popular languages for data processing and data science in general. The ecosystem provides a lot of libraries and frameworks that facilitate high-performance computing. Doing parallel programming in Python can prove quite tricky, though.
In this tutorial, we're going to study why parallelism is hard especially in the Python context, and for that, we will go through the following:
- Why is parallelism tricky in Python (hint: it's because of the GIL—the global interpreter lock).
- Threads vs. Processes: Different ways of achieving parallelism. When to use one over the other?
- Parallel vs. Concurrent: Why in some cases we can settle for concurrency rather than parallelism.
- Building a simple but practical example using the various techniques discussed.
Global Interpreter Lock
The Global Interpreter Lock (GIL) is one of the most controversial subjects in the Python world. In CPython, the most popular implementation of Python, the GIL is a mutex that makes things thread-safe. The GIL makes it easy to integrate with external libraries that are not thread-safe, and it makes non-parallel code faster. This comes at a cost, though. Due to the GIL, we can't achieve true parallelism via multithreading. Basically, two different native threads of the same process can't run Python code at once.
Things are not that bad, though, and here's why: stuff that happens outside the GIL realm is free to be parallel. In this category fall long-running tasks like I/O and, fortunately, libraries like numpy.
Threads vs. Processes
So Python is not truly multithreaded. But what is a thread? Let's take a step back and look at things in perspective.
A process is a basic operating system abstraction. It is a program that is in execution—in other words, code that is running. Multiple processes are always running in a computer, and they are executing in parallel.
A process can have multiple threads. They execute the same code belonging to the parent process. Ideally, they run in parallel, but not necessarily. The reason why processes aren't enough is because applications need to be responsive and listen for user actions while updating the display and saving a file.
If that's still a bit unclear, here's a cheatsheet:
| PROCESSES |
THREADS |
|---|---|
| Processes don't share memory |
Threads share memory |
| Spawning/switching processes is expensive |
Spawning/switching threads is less expensive |
| Processes require more resources |
Threads require fewer resources (are sometimes called lightweight processes) |
| No memory synchronisation needed |
You need to use synchronisation mechanisms to be sure you're correctly handling the data |
There isn’t one recipe that accommodates everything. Choosing one is greatly dependent on the context and the task you are trying to achieve.
Parallel vs. Concurrent
Now we'll go one step further and dive into concurrency. Concurrency is often misunderstood and mistaken for parallelism. That's not the case. Concurrency implies scheduling independent code to be executed in a cooperative manner. Take advantage of the fact that a piece of code is waiting on I/O operations, and during that time run a different but independent part of the code.
In Python, we can achieve lightweight concurrent behaviour via greenlets. From a parallelization perspective, using threads or greenlets is equivalent because neither of them runs in parallel. Greenlets are even less expensive to create than threads. Because of that, greenlets are heavily used for performing a huge number of simple I/O tasks, like the ones usually found in networking and web servers.
Now that we know the difference between threads and processes, parallel and concurrent, we can illustrate how different tasks are performed on the two paradigms. Here's what we're going to do: we will run, multiple times, a task outside the GIL and one inside it. We're running them serially, using threads and using processes. Let's define the tasks:
import os
import time
import threading
import multiprocessing
NUM_WORKERS = 4
def only_sleep():
""" Do nothing, wait for a timer to expire """
print("PID: %s, Process Name: %s, Thread Name: %s" % (
os.getpid(),
multiprocessing.current_process().name,
threading.current_thread().name)
)
time.sleep(1)
def crunch_numbers():
""" Do some computations """
print("PID: %s, Process Name: %s, Thread Name: %s" % (
os.getpid(),
multiprocessing.current_process().name,
threading.current_thread().name)
)
x = 0
while x < 10000000:
x += 1
We've created two tasks. Both of them are long-running, but only crunch_numbers actively performs computations. Let's run only_sleep serially, multithreaded and using multiple processes and compare the results:
## Run tasks serially
start_time = time.time()
for _ in range(NUM_WORKERS):
only_sleep()
end_time = time.time()
print("Serial time=", end_time - start_time)
# Run tasks using threads
start_time = time.time()
threads = [threading.Thread(target=only_sleep) for _ in range(NUM_WORKERS)]
[thread.start() for thread in threads]
[thread.join() for thread in threads]
end_time = time.time()
print("Threads time=", end_time - start_time)
# Run tasks using processes
start_time = time.time()
processes = [multiprocessing.Process(target=only_sleep()) for _ in range(NUM_WORKERS)]
[process.start() for process in processes]
[process.join() for process in processes]
end_time = time.time()
print("Parallel time=", end_time - start_time)
Here's the output I've got (yours should be similar, although PIDs and times will vary a bit):
PID: 95726, Process Name: MainProcess, Thread Name: MainThread PID: 95726, Process Name: MainProcess, Thread Name: MainThread PID: 95726, Process Name: MainProcess, Thread Name: MainThread PID: 95726, Process Name: MainProcess, Thread Name: MainThread Serial time= 4.018089056015015 PID: 95726, Process Name: MainProcess, Thread Name: Thread-1 PID: 95726, Process Name: MainProcess, Thread Name: Thread-2 PID: 95726, Process Name: MainProcess, Thread Name: Thread-3 PID: 95726, Process Name: MainProcess, Thread Name: Thread-4 Threads time= 1.0047411918640137 PID: 95728, Process Name: Process-1, Thread Name: MainThread PID: 95729, Process Name: Process-2, Thread Name: MainThread PID: 95730, Process Name: Process-3, Thread Name: MainThread PID: 95731, Process Name: Process-4, Thread Name: MainThread Parallel time= 1.014023780822754
Here are some observations:
-
In the case of the serial approach, things are pretty obvious. We're running the tasks one after the other. All four runs are executed by the same thread of the same process.
-
Using processes we cut the execution time down to a quarter of the original time, simply because the tasks are executed in parallel. Notice how each task is performed in a different process and on the
MainThreadof that process. -
Using threads we take advantage of the fact that the tasks can be executed concurrently. The execution time is also cut down to a quarter, even though nothing is running in parallel. Here's how that goes: we spawn the first thread and it starts waiting for the timer to expire. We pause its execution, letting it wait for the timer to expire, and in this time we spawn the second thread. We repeat this for all the threads. At one moment the timer of the first thread expires so we switch execution to it and we terminate it. The algorithm is repeated for the second and for all the other threads. At the end, the result is as if things were run in parallel. You'll also notice that the four different threads branch out from and live inside the same process:
MainProcess. -
You may even notice that the threaded approach is quicker than the truly parallel one. That's due to the overhead of spawning processes. As we noted previously, spawning and switching processes is an expensive operation.
Let's do the same routine but this time running the crunch_numbers task:
start_time = time.time()
for _ in range(NUM_WORKERS):
crunch_numbers()
end_time = time.time()
print("Serial time=", end_time - start_time)
start_time = time.time()
threads = [threading.Thread(target=crunch_numbers) for _ in range(NUM_WORKERS)]
[thread.start() for thread in threads]
[thread.join() for thread in threads]
end_time = time.time()
print("Threads time=", end_time - start_time)
start_time = time.time()
processes = [multiprocessing.Process(target=crunch_numbers) for _ in range(NUM_WORKERS)]
[process.start() for process in processes]
[process.join() for process in processes]
end_time = time.time()
print("Parallel time=", end_time - start_time)
Here's the output I've got:
PID: 96285, Process Name: MainProcess, Thread Name: MainThread PID: 96285, Process Name: MainProcess, Thread Name: MainThread PID: 96285, Process Name: MainProcess, Thread Name: MainThread PID: 96285, Process Name: MainProcess, Thread Name: MainThread Serial time= 2.705625057220459 PID: 96285, Process Name: MainProcess, Thread Name: Thread-1 PID: 96285, Process Name: MainProcess, Thread Name: Thread-2 PID: 96285, Process Name: MainProcess, Thread Name: Thread-3 PID: 96285, Process Name: MainProcess, Thread Name: Thread-4 Threads time= 2.6961309909820557 PID: 96289, Process Name: Process-1, Thread Name: MainThread PID: 96290, Process Name: Process-2, Thread Name: MainThread PID: 96291, Process Name: Process-3, Thread Name: MainThread PID: 96292, Process Name: Process-4, Thread Name: MainThread Parallel time= 0.8014059066772461
The main difference here is in the result of the multithreaded approach. This time it performs very similarly to the serial approach, and here's why: since it performs computations and Python doesn't perform real parallelism, the threads are basically running one after the other, yielding execution to one another until they all finish.
The Python Parallel/Concurrent Programming Ecosystem
Python has rich APIs for doing parallel/concurrent programming. In this tutorial we're covering the most popular ones, but you have to know that for any need you have in this domain, there's probably something already out there that can help you achieve your goal.
In the next section, we'll build a practical application in many forms, using all of the libraries presented. Without further ado, here are the modules/libraries we're going to cover:
-
threading: The standard way of working with threads in Python. It is a higher-level API wrapper over the functionality exposed by the_threadmodule, which is a low-level interface over the operating system's thread implementation. -
concurrent.futures: A module part of the standard library that provides an even higher-level abstraction layer over threads. The threads are modelled as asynchronous tasks. -
multiprocessing: Similar to thethreadingmodule, offering a very similar interface but using processes instead of threads. -
gevent and greenlets: Greenlets, also called micro-threads, are units of execution that can be scheduled collaboratively and can perform tasks concurrently without much overhead. -
celery: A high-level distributed task queue. The tasks are queued and executed concurrently using various paradigms likemultiprocessingorgevent.
Building a Practical Application
Knowing the theory is nice and fine, but the best way to learn is to build something practical, right? In this section, we're going to build a classic type of application going through all the different paradigms.
Let's build an application that checks the uptime of websites. There are a lot of such solutions out there, the most well-known ones being probably Jetpack Monitor and Uptime Robot. The purpose of these apps is to notify you when your website is down so that you can quickly take action. Here's how they work:
- The application goes very frequently over a list of website URLs and checks if those websites are up.
- Every website should be checked every 5-10 minutes so that the downtime is not significant.
- Instead of performing a classic HTTP GET request, it performs a HEAD request so that it does not affect your traffic significantly.
- If the HTTP status is in the danger ranges (400+, 500+), the owner is notified.
- The owner is notified either by email, text-message, or push notification.
Here's why it's essential to take a parallel/concurrent approach to the problem. As the list of websites grows, going through the list serially won't guarantee us that every website is checked every five minutes or so. The websites could be down for hours, and the owner won't be notified.
Let's start by writing some utilities:
# utils.py
import time
import logging
import requests
class WebsiteDownException(Exception):
pass
def ping_website(address, timeout=20):
"""
Check if a website is down. A website is considered down
if either the status_code >= 400 or if the timeout expires
Throw a WebsiteDownException if any of the website down conditions are met
"""
try:
response = requests.head(address, timeout=timeout)
if response.status_code >= 400:
logging.warning("Website %s returned status_code=%s" % (address, response.status_code))
raise WebsiteDownException()
except requests.exceptions.RequestException:
logging.warning("Timeout expired for website %s" % address)
raise WebsiteDownException()
def notify_owner(address):
"""
Send the owner of the address a notification that their website is down
For now, we're just going to sleep for 0.5 seconds but this is where
you would send an email, push notification or text-message
"""
logging.info("Notifying the owner of %s website" % address)
time.sleep(0.5)
def check_website(address):
"""
Utility function: check if a website is down, if so, notify the user
"""
try:
ping_website(address)
except WebsiteDownException:
notify_owner(address)
We'll actually need a website list to try our system out. Create your own list or use mine:
# websites.py
WEBSITE_LIST = [
'http://envato.com',
'http://amazon.co.uk',
'http://amazon.com',
'http://facebook.com',
'http://google.com',
'http://google.fr',
'http://google.es',
'http://google.co.uk',
'http://internet.org',
'http://gmail.com',
'http://ift.tt/gbk8l4',
'http://github.com',
'http://heroku.com',
'http://ift.tt/2oO8vlq',
'http://ift.tt/rBjKuM',
'http://rubyonrails.org',
'http://basecamp.com',
'http://trello.com',
'http://yiiframework.com',
'http://shopify.com',
'http://ift.tt/2oNXL6K',
'http://airbnb.com',
'http://instagram.com',
'http://snapchat.com',
'http://youtube.com',
'http://baidu.com',
'http://yahoo.com',
'http://live.com',
'http://linkedin.com',
'http://yandex.ru',
'http://netflix.com',
'http://wordpress.com',
'http://bing.com',
]
Normally, you'd keep this list in a database along with owner contact information so that you can contact them. Since this is not the main topic of this tutorial, and for the sake of simplicity, we're just going to use this Python list.
If you paid really good attention, you might have noticed two really long domains in the list that are not valid websites (I hope nobody bought them by the time you're reading this to prove me wrong!). I added these two domains to be sure we have some websites down on every run. Also, let's name our app UptimeSquirrel.
Serial Approach
First, let's try the serial approach and see how badly it performs. We'll consider this the baseline.
# serial_squirrel.py
import time
start_time = time.time()
for address in WEBSITE_LIST:
check_website(address)
end_time = time.time()
print("Time for SerialSquirrel: %ssecs" % (end_time - start_time))
# WARNING:root:Timeout expired for website http://ift.tt/2oO8vlq
# WARNING:root:Timeout expired for website http://ift.tt/2oNXL6K
# WARNING:root:Website http://bing.com returned status_code=405
# Time for SerialSquirrel: 15.881232261657715secs
Threading Approach
We're going to get a bit more creative with the implementation of the threaded approach. We're using a queue to put the addresses in and create worker threads to get them out of the queue and process them. We're going to wait for the queue to be empty, meaning that all the addresses have been processed by our worker threads.
# threaded_squirrel.py
import time
from queue import Queue
from threading import Thread
NUM_WORKERS = 4
task_queue = Queue()
def worker():
# Constantly check the queue for addresses
while True:
address = task_queue.get()
check_website(address)
# Mark the processed task as done
task_queue.task_done()
start_time = time.time()
# Create the worker threads
threads = [Thread(target=worker) for _ in range(NUM_WORKERS)]
# Add the websites to the task queue
[task_queue.put(item) for item in WEBSITE_LIST]
# Start all the workers
[thread.start() for thread in threads]
# Wait for all the tasks in the queue to be processed
task_queue.join()
end_time = time.time()
print("Time for ThreadedSquirrel: %ssecs" % (end_time - start_time))
# WARNING:root:Timeout expired for website http://ift.tt/2oO8vlq
# WARNING:root:Timeout expired for website http://ift.tt/2oNXL6K
# WARNING:root:Website http://bing.com returned status_code=405
# Time for ThreadedSquirrel: 3.110753059387207secs
concurrent.futures
As stated previously, concurrent.futures is a high-level API for using threads. The approach we're taking here implies using a ThreadPoolExecutor. We're going to submit tasks to the pool and get back futures, which are results that will be available to us in the future. Of course, we can wait for all futures to become actual results.
# future_squirrel.py
import time
import concurrent.futures
NUM_WORKERS = 4
start_time = time.time()
with concurrent.futures.ThreadPoolExecutor(max_workers=NUM_WORKERS) as executor:
futures = {executor.submit(check_website, address) for address in WEBSITE_LIST}
concurrent.futures.wait(futures)
end_time = time.time()
print("Time for FutureSquirrel: %ssecs" % (end_time - start_time))
# WARNING:root:Timeout expired for website http://ift.tt/2oO8vlq
# WARNING:root:Timeout expired for website http://ift.tt/2oNXL6K
# WARNING:root:Website http://bing.com returned status_code=405
# Time for FutureSquirrel: 1.812899112701416secs
The Multiprocessing Approach
The multiprocessing library provides an almost drop-in replacement API for the threading library. In this case, we're going to take an approach more similar to the concurrent.futures one. We're setting up a multiprocessing.Pool and submitting tasks to it by mapping a function to the list of addresses (think of the classic Python map function).
# multiprocessing_squirrel.py
import time
import socket
import multiprocessing
NUM_WORKERS = 4
start_time = time.time()
with multiprocessing.Pool(processes=NUM_WORKERS) as pool:
results = pool.map_async(check_website, WEBSITE_LIST)
results.wait()
end_time = time.time()
print("Time for MultiProcessingSquirrel: %ssecs" % (end_time - start_time))
# WARNING:root:Timeout expired for website http://ift.tt/2oO8vlq
# WARNING:root:Timeout expired for website http://ift.tt/2oNXL6K
# WARNING:root:Website http://bing.com returned status_code=405
# Time for MultiProcessingSquirrel: 2.8224599361419678secs
Gevent
Gevent is a popular alternative for achieving massive concurrency. There are a few things you need to know before using it:
-
Code performed concurrently by greenlets is deterministic. As opposed to the other presented alternatives, this paradigm guarantees that for any two identical runs, you'll always get the same results in the same order.
-
You need to monkey patch standard functions so that they cooperate with gevent. Here's what I mean by that. Normally, a socket operation is blocking. We're waiting for the operation to finish. If we were in a multithreaded environment, the scheduler would simply switch to another thread while the other one is waiting for I/O. Since we're not in a multithreaded environment, gevent patches the standard functions so that they become non-blocking and return control to the gevent scheduler.
To install gevent, run: pip install gevent
Here's how to use gevent to perform our task using a gevent.pool.Pool:
# green_squirrel.py
import time
from gevent.pool import Pool
from gevent import monkey
# Note that you can spawn many workers with gevent since the cost of creating and switching is very low
NUM_WORKERS = 4
# Monkey-Patch socket module for HTTP requests
monkey.patch_socket()
start_time = time.time()
pool = Pool(NUM_WORKERS)
for address in WEBSITE_LIST:
pool.spawn(check_website, address)
# Wait for stuff to finish
pool.join()
end_time = time.time()
print("Time for GreenSquirrel: %ssecs" % (end_time - start_time))
# Time for GreenSquirrel: 3.8395519256591797secs
Celery
Celery is an approach that mostly differs from what we've seen so far. It is battle tested in the context of very complex and high-performance environments. Setting up Celery will require a bit more tinkering than all the above solutions.
First, we'll need to install Celery:
pip install celery
Tasks are the central concepts within the Celery project. Everything that you'll want to run inside Celery needs to be a task. Celery offers great flexibility for running tasks: you can run them synchronously or asynchronously, real-time or scheduled, on the same machine or on multiple machines, and using threads, processes, Eventlet, or gevent.
The arrangement will be slightly more complex. Celery uses other services for sending and receiving messages. These messages are usually tasks or results from tasks. We're going to use Redis in this tutorial for this purpose. Redis is a great choice because it's really easy to install and configure, and it's really possible you already use it in your application for other purposes, such as caching and pub/sub.
You can install Redis by following the instructions on the Redis Quick Start page. Don't forget to install the redis Python library, pip install redis, and the bundle necessary for using Redis and Celery: pip install celery[redis].
Start the Redis server like this: $ redis-server
To get started building stuff with Celery, we'll first need to create a Celery application. After that, Celery needs to know what kind of tasks it might execute. To achieve that, we need to register tasks to the Celery application. We'll do this using the @app.task decorator:
# celery_squirrel.py
import time
from utils import check_website
from data import WEBSITE_LIST
from celery import Celery
from celery.result import ResultSet
app = Celery('celery_squirrel',
broker='redis://localhost:6379/0',
backend='redis://localhost:6379/0')
@app.task
def check_website_task(address):
return check_website(address)
if __name__ == "__main__":
start_time = time.time()
# Using `delay` runs the task async
rs = ResultSet([check_website_task.delay(address) for address in WEBSITE_LIST])
# Wait for the tasks to finish
rs.get()
end_time = time.time()
print("CelerySquirrel:", end_time - start_time)
# CelerySquirrel: 2.4979639053344727
Don't panic if nothing is happening. Remember, Celery is a service, and we need to run it. Till now, we only placed the tasks in Redis but did not start Celery to execute them. To do that, we need to run this command in the folder where our code resides:
celery worker -A do_celery --loglevel=debug --concurrency=4
Now rerun the Python script and see what happens. One thing to pay attention to: notice how we passed the Redis address to our Redis application twice. The broker parameter specifies where the tasks are passed to Celery, and backend is where Celery puts the results so that we can use them in our app. If we don't specify a result backend, there's no way for us to know when the task was processed and what the result was.
Also, be aware that the logs now are in the standard output of the Celery process, so be sure to check them out in the appropriate terminal.
Conclusions
I hope this has been an interesting journey for you and a good introduction to the world of parallel/concurrent programming in Python. This is the end of the journey, and there are some conclusions we can draw:
- There are several paradigms that help us achieve high-performance computing in Python.
- For the multi-threaded paradigm, we have the
threadingandconcurrent.futureslibraries. multiprocessingprovides a very similar interface tothreadingbut for processes rather than threads.- Remember that processes achieve true parallelism, but they are more expensive to create.
- Remember that a process can have more threads running inside it.
- Do not mistake parallel for concurrent. Remember that only the parallel approach takes advantage of multi-core processors, whereas concurrent programming intelligently schedules tasks so that waiting on long-running operations is done while in parallel doing actual computation.
Getting Started With Chart.js: Radar and Polar Area Charts
The previous tutorial of this series focused on creating line and bar charts using Chart.js. Both these charts have their own uses and configuration options that were covered in detail in the last tutorial.
In this tutorial, you will learn about two new chart types that can be created using Chart.js: radar and polar area charts. Just like the previous tutorial, we will start with a brief overview of the chart types and then move to a more detailed discussion.
Creating Radar Charts
Line and bar charts are useful when you want to compare only one or two properties of a large number of objects—for example, the population of all the countries in Asia or the number of different pollutants in the atmosphere.
Let's say you want to compare the density, opacity, viscosity, refractive index and boiling point of only three different liquids. Creating a bar chart for this data will be problematic as you will need to create five different columns for each liquid. It will also be hard to compare the corresponding properties of the liquids.
In situations where you have to compare a lot of properties of only a few objects, creating a radar chart is the most efficient method of visualizing and comparing data. These charts are also known as spider charts.
From Wikipedia, a radar chart is a graphical method of displaying multivariate data in the form of a two-dimensional chart of three or more quantitative variables represented on axes starting from the same point. The relative positions and angles of the axes are typically uninformative.
Let's create our first radar chart now. Radar charts are created by setting the type key in Chart.js to radar. Here is a very basic example.
var radarChart = new Chart(marksCanvas, {
type: 'radar',
data: marksData,
options: chartOptions
});
Let's plot the marks of two students of a class in five different subjects. Here is the code to provide the data for creating the chart.
var marksData = {
labels: ["English", "Maths", "Physics", "Chemistry", "Biology", "History"],
datasets: [{
label: "Student A",
backgroundColor: "rgba(200,0,0,0.2)",
data: [65, 75, 70, 80, 60, 80]
}, {
label: "Student B",
backgroundColor: "rgba(0,0,200,0.2)",
data: [54, 65, 60, 70, 70, 75]
}]
};
var radarChart = new Chart(marksCanvas, {
type: 'radar',
data: marksData
});
The first chart that we usually create does not have any background color specifically set by us. However, radar charts can have a lot of overlap, making it difficult to correctly identify the data points without any color coding.
For this reason, a color has been assigned to each dataset using the backgroundColor key. The following demo shows the final result of our code. As you can see, it is now very easy to compare the performance of both students in different subjects.
Besides the background color, you can also change the border color and border width for the chart using the borderColor and borderWidth keys. It is also possible for you to create dashed borders instead of continuous lines using the borderDash key. This key accepts an array as its value.
The first element of the array determines the length of the dashes, and the second element determines the space between them. You can also provide an offset value for drawing the dashes using the borderDashOffset key.
You can also control the border color and width for plotted points using the pointBorderColor and pointBorderWidth. Similarly, you can also set a background color for different points using the pointBackgroundColor key. The size of the plotted points can be specified using the pointRadius key. You can control the distance at which the points should start interacting with the mouse using the pointHitRadius key.
You can also control the appearance of the plotted points on hover using the pointHoverBackgroundColor, pointHoverBorderColor and pointHoverBorderWidth keys. One thing that you need to remember is that these hover keys will not wait for you to actually hover over the element to get triggered. The changes will take effect as soon as you are inside the hit radius set above.
There are a lot of shapes available for plotted points. They are circular by default. However, you can change the shape to triangle, rect, rectRounded, rectRot, cross, crossRot, star, line, and dash.
Let's use all these keys to redraw the previous radar chart. Here is the code to provide configuration options for the datasets and the scales.
var marksData = {
labels: ["English", "Maths", "Physics", "Chemistry", "Biology", "History"],
datasets: [{
label: "Student A",
backgroundColor: "transparent",
borderColor: "rgba(200,0,0,0.6)",
fill: false,
radius: 6,
pointRadius: 6,
pointBorderWidth: 3,
pointBackgroundColor: "orange",
pointBorderColor: "rgba(200,0,0,0.6)",
pointHoverRadius: 10,
data: [65, 75, 70, 80, 60, 80]
}, {
label: "Student B",
backgroundColor: "transparent",
borderColor: "rgba(0,0,200,0.6)",
fill: false,
radius: 6,
pointRadius: 6,
pointBorderWidth: 3,
pointBackgroundColor: "cornflowerblue",
pointBorderColor: "rgba(0,0,200,0.6)",
pointHoverRadius: 10,
data: [54, 65, 60, 70, 70, 75]
}]
};
var chartOptions = {
scale: {
ticks: {
beginAtZero: true,
min: 0,
max: 100,
stepSize: 20
},
pointLabels: {
fontSize: 18
}
},
legend: {
position: 'left'
}
};
Inside the chartOptions object, the min and max values are used to set the minimum and maximum values for the scale. The stepSize key can be used to tell Chart.js the number of steps that it should take to go from min to max. Here is the final result of the above code.
Creating Polar Area Charts
Polar area charts are similar to pie charts. Two major differences between these charts are that in polar area charts all the sectors have equal angles and the radius of each sector depends on the plotted value. These charts are used to plot cyclic phenomena. For example, you can use it to plot the number of migratory birds of a given species in your area in each season of the year.
The radius of each sector in these charts is proportional to the square root of the number of corresponding objects. In the case of migratory birds, the radius will be proportional to the square root of the number of birds in your area.
You can create polar area charts in Chart.js by setting the type key to polarArea. Here is the basic code that you need to create a polar chart.
var polarAreaChart = new Chart(birdsCanvas, {
type: 'polarArea',
data: birdsData,
options: chartOptions
});
Here is the code to plot the number of migratory birds on a polar area chart. The only data provided at this point is the number of birds and the background color for different seasons.
var birdsData = {
labels: ["Spring","Summer","Fall","Winter"],
datasets: [{
data: [1200, 1700, 800, 200],
backgroundColor: [
"rgba(255, 0, 0, 0.5)",
"rgba(100, 255, 0, 0.5)",
"rgba(200, 50, 255, 0.5)",
"rgba(0, 100, 255, 0.5)"
]
}]
};
var polarAreaChart = new Chart(birdsCanvas, {
type: 'polarArea',
data: birdsData
});
The above code will create the following polar area chart.
The polar area chart provides the usual options to set the backgroundColor, borderWidth, borderColor, hoverBackgroundColor, hoverBorderWidth, and hoverBorderColor. There are also some polar area chart specific keys that you can use to customize their appearance.
For example, you can set the starting angle for the first value in the dataset using the startAngle key. Similarly, the lineArc key that can be found under scale can be used to specify if the lines drawn should be circular or not by setting its value to true or false respectively.
The sectors drawn in the polar area chart are both rotated and scaled by default. However, you can prevent the scaling animation by setting the value of the animateScale key to false. Similarly, the rotating animation can be turned off by setting the value of the animateRotate key to false. Both these keys are available under animation.
The code below changes the value of a few keys to make the chart more visually appealing.
var birdsData = {
labels: ["Spring", "Summer", "Fall", "Winter"],
datasets: [{
data: [1200, 1700, 800, 200],
backgroundColor: [
"rgba(255, 0, 0, 0.6)",
"rgba(0, 255,200, 0.6)",
"rgba(200, 0, 200, 0.6)",
"rgba(0, 255, 0, 0.6)"
],
borderColor: "rgba(0, 0, 0, 0.8)"
}]
};
var chartOptions = {
startAngle: -Math.PI / 4,
legend: {
position: 'left'
},
animation: {
animateRotate: false
}
};
var polarAreaChart = new Chart(birdsCanvas, {
type: 'polarArea',
data: birdsData,
options: chartOptions
});
Besides rotating the chart and disabling the rotation animation, we have also moved the legend to the left of the chart by setting the value of position to left. This leaves enough space at the top of the chart to display it properly.
Final Thoughts
In this tutorial, you learned about the applications of radar and polar area charts. After that, you learned how to create basic charts and customize them with various configuration options available in Chart.js. You will learn about pie and bubble charts in the next part of the series.
If you're working with the web, especially on the front-end, JavaScript is important to know. Of course, it's not without its learning curves, and there are plenty of frameworks and libraries to keep you busy, as well. If you’re looking for additional resources to study or to use in your work, check out what we have available in the Envato marketplace.
If you have any questions about this tutorial, please let me know in the comments.
Wednesday, April 19, 2017
Working With PHP Arrays in the Right Way
In this tutorial, I am going to make a list of common PHP array functions with examples of usage and best practices. Every PHP developer must know how to use them and how to combine array functions to make code readable and short.
Also, there is a presentation with given code examples, so you can download it from the related links and show it to your colleagues to build a stronger team.
The Basics
Let's start with the basic functions that work with array keys and values. One of them is array_combine(), which creates an array using one array for keys and another for its values:
$keys = ['sky', 'grass', 'orange']; $values = ['blue', 'green', 'orange']; $array = array_combine($keys, $values); print_r($array); // Array // ( // [sky] => blue // [grass] => green // [orange] => orange // )
You should know, that the function array_values() returns an indexed array of values, array_keys() returns an array of keys of a given array, and array_flip() exchanges keys with values:
print_r(array_keys($array)); // ['sky', 'grass', 'orange'] print_r(array_values($array)); // ['blue', 'green', 'orange'] print_r(array_flip($array)); // Array // ( // [blue] => sky // [green] => grass // [orange] => orange // )
Make Your Code Shorter
The function list(), which is not really a function, but a language construction, is designed to assign variables in a short way. For example, here is a basic example of using the list() function:
// define array $array = ['a', 'b', 'c']; // without list() $a = $array[0]; $b = $array[1]; $c = $array[2]; // with list() list($a, $b, $c) = $array;
This construction works perfectly with functions like preg_slit() or explode() . Also, you can skip some parameters, if you don't need them to be defined:
$string = 'hello|wild|world';
list($hello, , $world) = explode('|', $string);
echo("$hello, $world"); // hello, world
Also, list() can be used with foreach, which makes this construction even better:
$arrays = [[1, 2], [3, 4], [5, 6]];
foreach ($arrays as list($a, $b)) {
$c = $a + $b;
echo($c . ', '); // 3, 7, 11,
}
With the extract() function, you can export an associative array to variables. For every element of an array, a variable will be created with the name of a key and value as a value of the element:
$array = [
'clothes' => 't-shirt',
'size' => 'medium',
'color' => 'blue',
];
extract($array);
echo("$clothes $size $color"); // t-shirt medium blue
Be aware that extract() is not safe if you are working with user data (like results of requests), so it is better to use this function with the flags EXTR_IF_EXISTS and EXTR_PREFIX_ALL.
The opposite of the previous function is the compact() function, which makes an associative array from variables:
$clothes = 't-shirt';
$size = 'medium';
$color = 'blue';
$array = compact('clothes', 'size', 'color');
print_r($array);
// Array
// (
// [clothes] => t-shirt
// [size] => medium
// [color] => blue
// )
Filtering Functions
There is a great function for array filtering, and it is called array_filter(). Pass the array as the first param and an anonymous function as the second param. Return true in a callback function if you want to leave this element in the array, and false if you don't:
$numbers = [20, -3, 50, -99, 55];
$positive = array_filter($numbers, function($number) {
return $number > 0;
});
print_r($positive); // [0 => 20, 2 => 50, 4 => 55]
There is a way to filter not only by the values. You can use ARRAY_FILTER_USE_KEY or ARRAY_FILTER_USE_BOTH as a third parameter to pass the key or both value and key to the callback function.
Also, you can call array_filter() without a callback to remove all empty values:
$numbers = [-1, 0, 1]; $not_empty = array_filter($numbers); print_r($not_empty); // [0 => -1, 2 => 1]
You can get only unique values from an array using the array_unique() function. Notice that the function will preserve the keys of the first unique elements:
$array = [1, 1, 1, 1, 2, 2, 2, 3, 4, 5, 5]; $uniques = array_unique($array); print_r($uniques); // Array // ( // [0] => 1 // [4] => 2 // [7] => 3 // [8] => 4 // [9] => 5 // )
With array_column(), you can get a list of column values from a multi-dimensional array, like an answer from a SQL database or an import from a CSV file. Just pass an array and column name:
$array = [
['id' => 1, 'title' => 'tree'],
['id' => 2, 'title' => 'sun'],
['id' => 3, 'title' => 'cloud'],
];
$ids = array_column($array, 'id');
print_r($ids); // [1, 2, 3]
Starting from PHP 7, array_column() becomes even more powerful, because it is now allowed to work with an array of objects. So working with an array of models just became easier:
$cinemas = Cinema::find()->all(); $cinema_ids = array_column($cinemas, 'id'); // php7 forever!
Walking Through the Arrays
Using array_map(), you can apply a callback to every element of an array. You can pass a function name or anonymous function to get a new array based on the given array:
$cities = ['Berlin', 'KYIV', 'Amsterdam', 'Riga'];
$aliases = array_map('strtolower', $cities);
print_r($aliases); // ['berlin', 'kyiv, 'warsaw', 'riga']
$numbers = [1, -2, 3, -4, 5];
$squares = array_map(function($number) {
return $number ** 2;
}, $numbers);
print_r($squares); // [1, 4, 9, 16, 25]
There is a myth that there is no way to pass values and keys of an array to a callback, but we can bust it:
$model = ['id' => 7, 'name'=>'James'];
$callback = function($key, $value) {
return "$key is $value";
};
$res = array_map($callback, array_keys($model), $model);
print_r($res);
// Array
// (
// [0] => id is 7
// [1] => name is James
// )
But this looks dirty. It is better to use array_walk() instead. This function looks the same as array_map(), but it works differently. First of all, an array is passed by a reference, so array_walk() doesn't create a new array, but changes a given array. So as a source array, you can pass the array value by a reference in a callback. Array keys can also be passed easily:
$fruits = [
'banana' => 'yellow',
'apple' => 'green',
'orange' => 'orange',
];
array_walk($fruits, function(&$value, $key) {
$value = "$key is $value";
});
print_r($fruits);
// Array
// (
// [banana] => banana is yellow
// [apple] => apple is green
// [orange] => orange is orange
// )
Joining the Arrays
The best way to merge two or more arrays in PHP is to use the array_merge() function. Items of arrays will be merged together, and values with the same string keys will be overwritten with the last value:
$array1 = ['a' => 'a', 'b' => 'b', 'c' => 'c']; $array2 = ['a' => 'A', 'b' => 'B', 'D' => 'D']; $merge = array_merge($array1, $array2); print_r($merge); // Array // ( // [a] => A // [b] => B // [c] => c // [D] => D // )
To remove array values from another array (or arrays), use array_diff(). To get values which are present in given arrays, use array_intersect(). The next examples will show how it works:
$array1 = [1, 2, 3, 4]; $array2 = [3, 4, 5, 6]; $diff = array_diff($array1, $array2); print_r($diff); // [0 => 1, 1 => 2] $intersect = array_intersect($array1, $array2); print_r($intersect); // [2 => 3, 3 => 4]
Do the Math With Array Values
Use array_sum() to get a sum of array values, array_product() to multiply them, or create your own formula with array_reduce():
$numbers = [1, 2, 3, 4, 5];
echo(array_sum($numbers)); // 15
echo(array_product($numbers)); // 120
echo(array_reduce($numbers, function($carry, $item) {
return $carry ? $carry / $item : 1;
})); // 0.0083 = 1/2/3/4/5
To count all the values of an array, use array_count_values(). It will give all unique values of a given array as keys and a count of these values as a value:
$things = ['apple', 'apple', 'banana', 'tree', 'tree', 'tree']; $values = array_count_values($things); print_r($values); // Array // ( // [apple] => 2 // [banana] => 1 // [tree] => 3 // )
Generating Arrays
To generate an array with a given size and the same value, use array_fill():
$bind = array_fill(0, 5, '?'); print_r($bind); // ['?', '?', '?', '?', '?']
To generate an array with a range in of keys and values, like day hours or letters, use range():
$letters = range('a', 'z');
print_r($letters); // ['a', 'b', ..., 'z']
$hours = range(0, 23);
print_r($hours); // [0, 1, 2, ..., 23]
To get a part of an array—for example, just the first three elements—use array_slice():
$numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]; $top = array_slice($numbers, 0, 3); print_r($top); // [1, 2, 3]
Sorting Arrays
It is good to remember that every sorting function in PHP works with arrays by a reference and returns true on success or false on failure. There's a basic sorting function called sort(), and it sorts values in ascending order without preserving keys. The sorting function can be prepended by the following letters:
- a, sort preserving keys
- k, sort by keys
- r, sort in reverse/descending order
- u, sort with a user function
You can see the combinations of these letters in the following table:
| a | k | r | u | |
| a | asort | arsort | uasort | |
| k | ksort | krsort | ||
| r | arsort | krsort | rsort | |
| u | uasort | usort |
Combining Array Functions Like a Boss
The real magic begins when you start to combine array functions. Here is how you can trim and remove empty values in just a single line of code with array_filter() and array_map():
$values = ['say ', ' bye', ' ', ' to', ' spaces ', ' '];
$words = array_filter(array_map('trim', $values));
print_r($words); // ['say', 'bye', 'to', 'spaces']
To create an id to a title map from an array of models, we can use a combination of array_combine() and array_column():
$models = [$model1, $model2, $model3];
$id_to_title = array_combine(
array_column($models, 'id'),
array_column($models, 'title')
);
To get the top three values of an array, we can use array_count_values(), arsort(), and array_slice():
$letters = ['a', 'a', 'a', 'a', 'b', 'b', 'c', 'd', 'd', 'd', 'd', 'd']; $values = array_count_values($letters); // get key to count array arsort($values); // sort descending preserving key $top = array_slice($values, 0, 3); // get top 3 print_r($top); // Array // ( // [d] => 5 // [a] => 4 // [b] => 2 // )
It is easy to use array_sum() and array_map() to calculate the sum of order in a few rows:
$order = [
['product_id' => 1, 'price' => 99, 'count' => 1],
['product_id' => 2, 'price' => 50, 'count' => 2],
['product_id' => 2, 'price' => 17, 'count' => 3],
];
$sum = array_sum(array_map(function($product_row) {
return $product_row['price'] * $product_row['count'];
}, $order));
print_r($sum); // 250
Conclusion
As you can see, knowledge of the main array functions can make your code much shorter and more readable. Of course, PHP has many more array functions, and even the given functions have many variations to use with extra parameters and flags, but I think that in this tutorial we've covered the basics that every PHP developer should know.
Please note that I've created a presentation with the given examples, so you can download it from the related links and show it to your team.
If you have any questions, don't hesitate to ask them in the comments to the article.
Further Reading and Related Links
- Arrays Manual on php.net
- Anonymous functions (closures) on php.net
- Download a presentation for your team
