Monday, April 30, 2018
Regular Expressions With Go: Part 2
Overview
This is part two of a two-part series of tutorials about regular expressions in Go. In part one we learned what regular expressions are, how to express them in Go, and the basics of using the Go regexp library to match text against regular expression patterns.
In part two, we will focus on using the regexp library to its full extent, including compiling regular expressions, finding one or more matches in the text, replacing regular expressions, grouping submatches, and dealing with new lines.
Using the Regexp Library
The regexp library provides full-fledged support for regular expressions as well as the ability to compile your patterns for more efficient execution when using the same pattern to match against multiple texts. You can also find indices of matches, replace matches, and use groups. Let's dive in.
Compiling Your Regex
There are two methods for compiling regexes: Compile()
and MustCompile()
. Compile()
will return an error if the provided pattern is invalid. MustCompile()
will panic. Compilation is recommended if you care about performance and plan to use the same regex multiple times. Let's change our match()
helper function to take a compiled regex. Note that there is no need to check for errors because the compiled regex must be valid.
func match(r *regexp.Regexp, text string) { matched := r.MatchString(text) if matched { fmt.Println("√", r.String(), ":", text) } else { fmt.Println("X", r.String(), ":", text) } }
Here is how to compile and use the same compiled regex multiple times:
func main() { es := `(\bcats?\b)|(\bdogs?\b)|(\brats?\b)` e := regexp.MustCompile(es) match(e, "It's raining dogs and cats") match(e, "The catalog is ready. It's hotdog time!") match(e, "It's a dog eat dog world.") } Output: √ (\bcats?\b)|(\bdogs?\b)|(\brats?\b) : It's raining dogs and cats X (\bcats?\b)|(\bdogs?\b)|(\brats?\b) : The catalog is ready. It's hotdog time! √ (\bcats?\b)|(\bdogs?\b)|(\brats?\b) : It's a dog eat dog world.
Finding
The Regexp object has a lot of FindXXX()
methods. Some of them return the first match, others return all matches, and yet others return an index or indexes. Interestingly enough, the names of all 16 methods of functions match the following regex: Find(All)?(String)?(Submatch)?(Index)?
If 'All' is present then all matches are returned vs. the leftmost one. If 'String' is present then the target text and the return values are strings vs. byte arrays. If 'Submatch' is present then submatches (groups) are returned vs. just simple matches. If 'Index' is present then indexes within the target text are returned vs. the actual matches.
Let's take one of the more complex functions to task and use the FindAllStringSubmatch()
method. It takes a string and a number n
. If n
is -1, it will return all matching indices. If n is a non-negative integer then it will return the n leftmost matches. The result is a slice of string slices.
The result of each submatch is the full match followed by the captured group. For example, consider a list of names where some of them have titles such "Mr.", "Mrs.", or "Dr.". Here is a regex that captures the title as a submatch and then the rest of the name after a space: \b(Mr\.|Mrs\.|Dr\.) .*
.
func main() { re := regexp.MustCompile(`\b(Mr\.|Mrs\.|Dr\.) .*`) fmt.Println(re.FindAllStringSubmatch("Dr. Dolittle", -1)) fmt.Println(re.FindAllStringSubmatch(`Mrs. Doubtfire Mr. Anderson`, -1)) } Output: [[Dr. Dolittle Dr.]] [[Mrs. Doubtfire Mrs.] [Mr. Anderson Mr.]]
As you can see in the output, the full match is captured first and then just the title. For each line, the search resets.
Replacing
Finding matches is great, but often you may need to replace the match with something else. The Regexp object has several ReplaceXXX()
methods as usual for dealing with strings vs. byte arrays and literal replacements vs. expansions. In the great book 1984 by George Orwell, the slogans of the party are inscribed on the white pyramid of the ministry of truth:
- War is Peace
- Freedom is Slavery
- Ignorance is Strength
I found a little essay on The Price of Freedom that uses some of these terms. Let's correct a snippet of it according to the party doublespeak using Go regexes. Note that some of the target words for replacement use different capitalization. The solution is to add the case-insensitive flag (i?)
at the beginning of the regex.
Since the translation is different depending on the case, we need a more sophisticated approach then literal replacement. Luckily (or by design), the Regexp object has a replace method that accepts a function it uses to perform the actual replacement. Let's define our replacer function that returns the translation with the correct case.
func replacer(s string) string { d := map[string]string{ "war": "peace", "WAR": "PEACE", "War": "Peace", "freedom": "slavery", "FREEDOM": "SLAVERY", "Freedom": "Slavery", "ignorance": "strength", "IGNORANCE": "STRENGTH", "Ignorance": "Strength", } r, ok := d[s] if ok { return r } else { return s } }
Now, we can perform the actual replacement:
func main() { text := `THE PRICE OF FREEDOM: Americans at War Americans have gone to war to win their independence, expand their national boundaries, define their freedoms, and defend their interests around the globe.` expr := `(?i)(war|freedom|ignorance)` r := regexp.MustCompile(expr) result := r.ReplaceAllStringFunc(text, replacer) fmt.Println(result) } Output: THE PRICE OF SLAVERY: Americans at Peace Americans have gone to peace to win their independence, expand their national boundaries, define their slaverys, and defend their interests around the globe.
The output is somewhat incoherent, which is the hallmark of good propaganda.
Grouping
We saw how to use grouping with submatches earlier. But it is sometimes difficult to handle multiple submatches. Named groups can help a lot here. Here is how to name your submatch groups and populate a dictionary for easy access by name:
func main() { e := `(?P<first>\w+) (?P<middle>.+ )?(?P<last>\w+)` r := regexp.MustCompile(e) names := r.SubexpNames() fullNames := []string{ `John F. Kennedy`, `Michael Jordan`} for _, fullName := range fullNames { result := r.FindAllStringSubmatch(fullName, -1) m := map[string]string{} for i, n := range result[0] { m[names[i]] = n } fmt.Println("first name:", m["first"]) fmt.Println("middle_name:", m["middle"]) fmt.Println("last name:", m["last"]) fmt.Println() } } Output: first name: John middle_name: F. last name: Kennedy first name: Michael middle_name: last name: Jordan
Dealing With New Lines
If you remember, I said that the dot special character matches any character. Well, I lied. It doesn't match the newline (\n
) character by default. That means that your matches will not cross lines unless you specify it explicitly with the special flag (?s)
that you can add to the beginning of your regex. Here is an example with and without the flag.
func main() { text := "1111\n2222" expr := []string{".*", "(?s).*"} for _, e := range expr { r := regexp.MustCompile(e) result := r.FindString(text) result = strings.Replace(result, "\n", `\n`, -1) fmt.Println(e, ":", result) fmt.Println() } } Output: .* : 1111 (?s).* : 1111\n2222
Another consideration is whether to treat the ^
and $
special characters as the beginning and end of the whole text (the default) or as the beginning and end of each line with the (?m)
flag.
Conclusion
Regular expressions are a powerful tool when working with semi-structured text. You can use them to validate textual input, clean it up, transform it, normalize it, and in general deal with a lot of diversity using concise syntax.
Go provides a library with an easy-to-use interface that consists of a Regexp object with many methods. Give it a try, but beware of the pitfalls.
Sunday, April 29, 2018
Saturday, April 28, 2018
Friday, April 27, 2018
Translating Stimulus Apps With I18Next
In my previous article I covered Stimulus—a modest JavaScript framework created by Basecamp. Today I'll talk about internationalizing a Stimulus application, since the framework does not provide any I18n tools out of the box. Internationalization is an important step, especially when your app is used by people from all around the world, so a basic understanding of how to do it may really come in handy.
Of course, it is up to you to decide which internationalization solution to implement, be it jQuery.I18n, Polyglot, or some other. In this tutorial I would like to show you a popular I18n framework called I18next that has lots of cool features and provides many additional third-party plugins to simplify the development process even further. Even with all these features, I18next is not a complex tool, and you don't need to study lots of documentation to get started.
In this article, you will learn how to enable I18n support in Stimulus applications with the help of the I18next library. Specifically, we'll talk about:
- I18next configuration
- translation files and loading them asynchronously
- performing translations and translating the whole page in one go
- working with plurals and gender information
- switching between locales and persisting the chosen locale in the GET parameter
- setting locale based on the user's preferences
The source code is available in the tutorial GitHub repo.
Bootstrapping a Stimulus App
In order to get started, let's clone the Stimulus Starter project and install all the dependencies using the Yarn package manager:
git clone https://github.com/stimulusjs/stimulus-starter.git cd stimulus-starter yarn install
We're going to build a simple web application that loads information about the registered users. For each user, we'll display his/her login and the number of photos he or she has uploaded so far (it does not really matter what these photos are).
Also, we are going to present a language switcher at the top of the page. When a language is chosen, the interface should be translated right away without page reloads. Moreover, the URL should be appended with a ?locale
GET parameter specifying which locale is currently being utilized. Of course, if the page is loaded with this parameter already provided, the proper language should be set automatically.
Okay, let's proceed to rendering our users. Add the following line of code to the public/index.html file:
<div data-controller="users" data-users-url="/api/users/index.json"></div>
Here, we are using the users
controller and providing a URL from which to load our users. In a real-world application, we would probably have a server-side script that fetches users from the database and responds with JSON. For this tutorial, however, let's simply place all the necessary data into the public/api/users/index.json file:
[ { "login": "johndoe", "photos_count": "15", "gender": "male" }, { "login": "annsmith", "photos_count": "20", "gender": "female" } ]
Now create a new src/controllers/users_controller.js file:
import { Controller } from "stimulus" export default class extends Controller { connect() { this.loadUsers() } }
As soon as the controller is connected to the DOM, we are asynchronously loading our users with the help of the loadUsers()
method:
loadUsers() { fetch(this.data.get("url")) .then(response => response.text()) .then(json => { this.renderUsers(json) }) }
This method sends a fetch request to the given URL, grabs the response, and finally renders the users:
renderUsers(users) { let content = '' JSON.parse(users).forEach((user) => { content += `<div>Login: ${user.login}<br>Has uploaded ${user.photos_count} photo(s)</div><hr>` }) this.element.innerHTML = content }
renderUsers()
, in turn, parses JSON, constructs a new string with all the content, and lastly displays this content on the page (this.element
is going to return the actual DOM node that the controller is connected to, which is div
in our case).
I18next
Now we are going to proceed to integrating I18next into our app. Add two libraries to our project: I18next itself and a plugin to enable asynchronous loading of translation files from the back end:
yarn add i18next i18next-xhr-backend
We are going to store all I18next-related stuff in a separate src/i18n/config.js file, so create it now:
import i18next from 'i18next' import I18nXHR from 'i18next-xhr-backend' const i18n = i18next.use(I18nXHR).init({ fallbackLng: 'en', whitelist: ['en', 'ru'], preload: ['en', 'ru'], ns: 'users', defaultNS: 'users', fallbackNS: false, debug: true, backend: { loadPath: '/i18n//.json', } }, function(err, t) { if (err) return console.error(err) }); export { i18n as i18n }
Let's go from top to bottom to understand what's going on here:
use(I18nXHR)
enables the i18next-xhr-backend plugin.fallbackLng
tells it to use English as a fallback language.whitelist
allows only English and Russian languages to be set. Of course, you may choose any other languages.preload
instructs translation files to be preloaded from the server, rather than loading them when the corresponding language is selected.ns
means "namespace" and accepts either a string or an array. In this example we have only one namespace, but for larger applications you may introduce other namespaces, likeadmin
,cart
,profile
, etc. For each namespace, a separate translation file should be created.defaultNS
setsusers
to be the default namespace.fallbackNS
disables namespace fallback.debug
allows debugging information to be displayed in the browser's console. Specifically, it says which translation files are loaded, which language is selected, etc. You will probably want to disable this setting before deploying the application to production.backend
provides configuration for the I18nXHR plugin and specifies where to load translations from. Note that the path should contain the locale's title, whereas the file should be named after the namespace and have the .json extensionfunction(err, t)
is the callback to run when I18next is ready (or when an error was raised).
Next, let's craft translation files. Translations for the Russian language should be placed into the public/i18n/ru/users.json file:
{ "login": "Логин" }
login
here is the translation key, whereas Логин
is the value to display.
English translations, in turn, should go to the public/i18n/en/users.json file:
{ "login": "Login" }
To make sure that I18next works, you may add the following line of code to the callback inside the i18n/config.js file:
// config goes here... function(err, t) { if (err) return console.error(err) console.log(i18n.t('login')) }
Here, we are using a method called t
that means "translate". This method accepts a translation key and returns the corresponding value.
However, we may have many parts of the UI that need to be translated, and doing so by utilizing the t
method would be quite tedious. Instead, I suggest that you use another plugin called loc-i18next that allows you to translate multiple elements at once.
Translating in One Go
Install the loc-i18next plugin:
yarn add loc-i18next
Import it at the top of the src/i18n/config.js file:
import locI18next from 'loc-i18next'
Now provide the configuration for the plugin itself:
// other config const loci18n = locI18next.init(i18n, { selectorAttr: 'data-i18n', optionsAttr: 'data-i18n-options', useOptionsAttr: true }); export { loci18n as loci18n, i18n as i18n }
There are a couple of things to note here:
locI18next.init(i18n)
creates a new instance of the plugin based on the previously defined instance of I18next.selectorAttr
specifies which attribute to use to detect elements that require localization. Basically, loc-i18next is going to search for such elements and use the value of thedata-i18n
attribute as the translation key.optionsAttr
specifies which attribute contains additional translation options.useOptionsAttr
instructs the plugin to use the additional options.
Our users are being loaded asynchronously, so we have to wait until this operation is done and only perform localization after that. For now, let's simply set a timer that should wait for two seconds before calling the localize()
method—that's a temporary hack, of course.
import { loci18n } from '../i18n/config' // other code... loadUsers() { fetch(this.data.get("url")) .then(response => response.text()) .then(json => { this.renderUsers(json) setTimeout(() => { // <--- this.localize() }, '2000') }) }
Code the localize()
method itself:
localize() { loci18n('.users') }
As you see, we only need to pass a selector to the loc-i18next plugin. All elements inside (that have the data-i18n
attribute set) will be localized automatically.
Now tweak the renderUsers
method. For now, let's only translate the "Login" word:
renderUsers(users) { let content = '' JSON.parse(users).forEach((user) => { content += `<div class="users">ID: ${user.id}<br><span data-i18n="login"></span>: ${user.login}<br>Has uploaded ${user.photos_count} photo(s)</div><hr>` }) this.element.innerHTML = content }
Nice! Reload the page, wait for two seconds, and make sure that the "Login" word appears for each user.
Plurals and Gender
We have localized part of the interface, which is really cool. Still, each user has two more fields: the number of uploaded photos and gender. Since we can't predict how many photos each user is going to have, the "photo" word should be pluralized properly based on the given count. In order to do this, we'll require a data-i18n-options
attribute configured previously. To provide the count, data-i18n-options
should be assigned with the following object: { "count": YOUR_COUNT }
.
Gender information should be taken into consideration as well. The word "uploaded" in English can be applied to both male and female, but in Russian it becomes either "загрузил" or "загрузила", so we need data-i18n-options
again, which has { "context": "GENDER" }
as a value. Note, by the way, that you can employ this context to achieve other tasks, not only to provide gender information.
renderUsers(users) { let content = '' JSON.parse(users).forEach((user) => { content += `<div class="users"><span data-i18n="login"></span>: ${user.login}<br><span data-i18n="uploaded" data-i18n-options="{ 'context': '${user.gender}' }"></span> <span data-i18n="photos" data-i18n-options="{ 'count': ${user.photos_count} }"></span></div><hr>` }) this.element.innerHTML = content }
Now update the English translations:
{ "login": "Login", "uploaded": "Has uploaded", "photos": "one photo", "photos_plural": " photos" }
Nothing complex here. Since for English we don't care about the gender information (which is the context), the translation key should be simply uploaded
. To provide properly pluralized translations, we are using the photos
and photos_plural
keys. The part is interpolation and will be replaced with the actual number.
As for the Russian language, things are more complex:
{ "login": "Логин", "uploaded_male": "Загрузил уже", "uploaded_female": "Загрузила уже", "photos_0": "одну фотографию", "photos_1": " фотографии", "photos_2": " фотографий" }
First of all, note that we have both uploaded_male
and uploaded_female
keys for two possible contexts. Next, pluralization rules are also more complex in Russian than in English, so we have to provide not two, but three possible phrases. I18next supports many languages out of the box, and this small tool can help you to understand which pluralization keys should be specified for a given language.
Switching Locale
We are done with translating our application, but users should be able to switch between locales. Therefore, add a new "language switcher" component to the public/index.html file:
<ul data-controller="languages" class="language-switcher"></ul>
Craft the corresponding controller inside the src/controllers/languages_controller.js file:
import { Controller } from "stimulus" import { i18n, loci18n } from '../i18n/config' export default class extends Controller { initialize() { let languages = [ {title: 'English', code: 'en'}, {title: 'Русский', code: 'ru'} ] this.element.innerHTML = languages.map((lang) => { return `<li data-action="click->languages#switchLanguage" data-lang="${lang.code}">${lang.title}</li>` }).join('') } }
Here we are using the initialize()
callback to display a list of supported languages. Each li
has a data-action
attribute which specifies what method (switchLanguage
, in this case) should be triggered when the element is clicked on.
Now add the switchLanguage()
method:
switchLanguage(e) { this.currentLang = e.target.getAttribute("data-lang") }
It simply takes the target of the event and grabs the value of the data-lang
attribute.
I would also like to add a getter and setter for the currentLang
attribute:
get currentLang() { return this.data.get("currentLang") } set currentLang(lang) { if(i18n.language !== lang) { i18n.changeLanguage(lang) } if(this.currentLang !== lang) { this.data.set("currentLang", lang) loci18n('body') this.highlightCurrentLang() } }
The getter is very simple—we fetch the value of the currently used language and return it.
The setter is more complex. First of all, we use the changeLanguage
method if the currently set language is not equal to the selected one. Also, we are storing the newly selected locale under the data-current-lang
attribute (which is accessed in the getter), localizing the body of the HTML page using the loc-i18next plugin, and lastly highlighting the currently used locale.
Let's code the highlightCurrentLang()
:
highlightCurrentLang() { this.switcherTargets.forEach((el, i) => { el.classList.toggle("current", this.currentLang === el.getAttribute("data-lang")) }) }
Here we are iterating over an array of locale switchers and comparing the values of their data-lang
attributes to the value of the currently used locale. If the values match, the switcher is assigned with a current
CSS class, otherwise this class is removed.
To make the this.switcherTargets
construct work, we need to define Stimulus targets in the following way:
static targets = [ "switcher" ]
Also, add data-target
attributes with values of switcher
for the li
s:
initialize() { // ... this.element.innerHTML = languages.map((lang) => { return `<li data-action="click->languages#switchLanguage" data-target="languages.switcher" data-lang="${lang.code}">${lang.title}</li>` }).join('') // ... }
Another important thing to consider is that translation files may take some time to load, and we must wait for this operation to complete before allowing the locale to be switched. Therefore, let's take advantage of the loaded
callback:
initialize() { i18n.on('loaded', (loaded) => { // <--- let languages = [ {title: 'English', code: 'en'}, {title: 'Русский', code: 'ru'} ] this.element.innerHTML = languages.map((lang) => { return `<li data-action="click->languages#switchLanguage" data-target="languages.switcher" data-lang="${lang.code}">${lang.title}</li>` }).join('') this.currentLang = i18n.language }) }
Lastly, don't forget to remove setTimeout
from the loadUsers()
method:
loadUsers() { fetch(this.data.get("url")) .then(response => response.text()) .then(json => { this.renderUsers(json) this.localize() }) }
Persisting Locale in the URL
After the locale is switched, I would like to add a ?lang
GET parameter to the URL containing the code of the chosen language. Appending a GET param without reloading the page can be easily done with the help of the History API:
set currentLang(lang) { if(i18n.language !== lang) { i18n.changeLanguage(lang) window.history.pushState(null, null, `?lang=${lang}`) // <--- } if(this.currentLang !== lang) { this.data.set("currentLang", lang) loci18n('body') this.highlightCurrentLang() } }
Detecting Locale
The last thing we are going to implement today is the ability to set the locale based on the user's preferences. A plugin called LanguageDetector can help us to solve this task. Add a new Yarn package:
yarn add i18next-browser-languagedetector
Import LanguageDetector
inside the i18n/config.js file:
import LngDetector from 'i18next-browser-languagedetector'
Now tweak the configuration:
const i18n = i18next.use(I18nXHR).use(LngDetector).init({ // <--- // other options go here... detection: { order: ['querystring', 'navigator', 'htmlTag'], lookupQuerystring: 'lang', } }, function(err, t) { if (err) return console.error(err) });
The order
option lists all the techniques (sorted by their importance) that the plugin should try in order to "guess" the preferred locale:
querystring
means checking a GET param containing the locale's code.lookupQuerystring
sets the name of the GET param to use, which islang
in our case.navigator
means getting locale data from the user's request.htmlTag
involves fetching the preferred locale from thelang
attribute of thehtml
tag.
Conclusion
In this article we have taken a look at I18next—a popular solution to translate JavaScript applications with ease. You have learned how to integrate I18next with the Stimulus framework, configure it, and load translation files in an asynchronous manner. Also, you've seen how to switch between locales and set the default language based on the user's preferences.
I18next has some additional configuration options and many plugins, so be sure to browse its official documentation to learn more. Also note that Stimulus does not force you to use a specific localization solution, so you may also try using something like jQuery.I18n or Polyglot.
That's all for today! Thank you for reading along, and until the next time.
Regular Expressions With Go: Part 1
Overview
Regular expressions (AKA regex) are a formal language that defines a sequence of characters with some pattern. In the real world they can be used to solve a lot of problems with semi-structured text. You can extract the important bits and pieces from text with a lot of decorations or unrelated content. Go has a strong regex package in its standard library that lets you slice and dice text with regexes.
In this two-part series, you'll learn what regular expressions are and how to use regular expressions effectively in Go to accomplish many common tasks. If you're not familiar with regular expressions at all, there are lots of great tutorials. Here is a good one.
Understanding Regular Expressions
Let's start with a quick example. You have some text, and you want to check if it contains an email address. An email address is specified rigorously in RFC 822. In short, it has a local part followed by an @ symbol followed by a domain. The mail address will be separated from the rest of the text by space.
To figure out if it contains an email address, the following regex will do: ^\w+@\w+\.\w+$
. Note that this regex is a little permissive and will allow some invalid email addresses through. But it's good enough to demonstrate the concept. Let's try it on a couple of potential email addresses before explaining how it works:
package main import ( "os" "regexp" "fmt" ) func check(err error) { if err != nil { fmt.Println(err.Error()) os.Exit(1) } } func main() { emails := []string{ "brown@fox", "brown@fox.", "brown@fox.com", "br@own@fox.com", } pattern := `^\w+@\w+\.\w+$` for _, email := range emails { matched, err := regexp.Match(pattern, []byte(email)) check(err) if matched { fmt.Printf("√ '%s' is a valid email\n", email) } else { fmt.Printf("X '%s' is not a valid email\n", email) } } } Output: X 'brown@fox' is not a valid email X 'brown@fox.' is not a valid email √ 'brown@fox.com' is a valid email X 'br@own@fox.com' is not a valid email
Our regular expression works on this little sample. The first two addresses were rejected because the domain didn't have a dot or didn't have any characters after the dot. The third email was formatted correctly. The last candidate had two @ symbols.
Let's break this regex down: ^\w+@\w+\.\w+$
Character/Symbol | Meaning |
---|---|
^ | Beginning of the target text |
\w | Any word characters [0-9A-Za-z_] |
+ | At least one of the previous characters |
@ | Literally the @ character |
\. | The literal dot character. Must be escaped with \ |
$ | End of target text |
Altogether, this regex will match pieces of text that start with one or more word characters, followed by the "@" character, followed again by one or more word characters, followed by a dot and followed by yet again one or more word characters.
Dealing With Special Characters
The following characters have special meanings in regular expressions: .+*?()|[]{}^$\
. We have already saw many of them in the email example. If we want to match them literally, we need to escape them with a backslash. Let's introduce a little helper function called match()
that will save us a lot of typing. It takes a pattern and some text, uses the regexp.Match()
method to match the pattern to the text (after converting the text to a byte array), and prints the results:
func match(pattern string, text string) { matched, _ := regexp.Match(pattern, []byte(text)) if matched { fmt.Println("√", pattern, ":", text) } else { fmt.Println("X", pattern, ":", text) } }
Here's an example of matching a regular character like z
vs. matching a special character like ?
:
func main() { text := "Can I haz cheezburger?" pattern := "z" match(pattern, text) pattern = "\\?" match(pattern, text) pattern = `\?` match(pattern, text) } Output: √ z : Can I haz cheezburger? √ \? : Can I haz cheezburger? √ \? : Can I haz cheezburger?
The regex pattern \?
contains a backslash that must be escaped with another backslash when represented as a regular Go string. The reason is that backslash is also used to escape special characters in Go strings like newline (\n
). If you want to match the backslash character itself, you'll need four slashes!
The solution is to use Go raw strings with the backtick (`
) instead of double quotes. Of course, if you want to match the newline character, you must go back to regular strings and deal with multiple backslash escapes.
Placeholders and Repetitions
In most cases, you don't try to literally match a sequence of specific characters like "abc", but a sequence of unknown length with maybe some known characters injected somewhere. Regexes support this use case with the dot .
special character that stands for any character whatsoever. The *
special character repeats the previous character (or group) zero or more times. If you combine them, as in .*
, then you match anything because it simply means zero or more characters. The +
is very similar to *
, but it matches one or more of the previous characters or groups. So .+
will match any non-empty text.
Using Boundaries
There are three types of boundaries: the start of the text denoted by ^
, the end of the text denoted by $
, and the word boundary denoted by \b
. For example, consider this text from the classic movie The Princess Bride: "My name is Inigo Montoya. You killed my father. Prepare to die." If you match just "father" you get a match, but if you're looking for "father" at the end of the text, you need to add the $
character, and then there will be no match. On the other hand, matching "Hello" at the beginning works well.
func main() { text := "Hello, my name is Inigo Montoya, you killed my father, prepare to die." pattern := "father" match(pattern, text) pattern = "father$" match(pattern, text) pattern = "^Hello" match(pattern, text) } Output: √ father : Hello, my name is Inigo Montoya, you killed my father, prepare to die. X father$ : Hello, my name is Inigo Montoya, you killed my father, prepare to die. √ ^Hello : Hello, my name is Inigo Montoya, you killed my father, prepare to die.
Word boundaries look at each word. You can start and/or end a pattern with the \b
. Note that punctuation marks like commas are considered a boundary and not part of the word. Here are a few examples:
func main() { text := `Hello, my name is Inigo Montoya, you killed my father, prepare to die.` pattern := `kill` match(pattern, text) pattern = `\bkill` match(pattern, text) pattern = `kill\b` match(pattern, text) pattern = `\bkill\b` match(pattern, text) pattern = `\bkilled\b` match(pattern, text) pattern = `\bMontoya,\b` match(pattern, text) } Output: √ kill : Hello, my name is Inigo Montoya, you killed my father, prepare to die. √ \bkill : Hello, my name is Inigo Montoya, you killed my father, prepare to die. X kill\b : Hello, my name is Inigo Montoya, you killed my father, prepare to die. X \bkill\b : Hello, my name is Inigo Montoya, you killed my father, prepare to die. √ \bkilled\b : Hello, my name is Inigo Montoya, you killed my father, prepare to die. X \bMontoya,\b : Hello, my name is Inigo Montoya, you killed my father, prepare to die.
Using Classes
It's often useful to treat all groups of characters together like all digits, whitespace characters, or all alphanumeric characters. Golang supports the POSIX classes, which are:
Character Class | Meaning |
---|---|
[:alnum:] |
alphanumeric (≡ [0-9A-Za-z]) |
[:alpha:] |
alphabetic (≡ [A-Za-z]) |
[:ascii:] |
ASCII (≡ [\x00-\x7F]) |
[:blank:] |
blank (≡ [\t ]) |
[:cntrl:] |
control (≡ [\x00-\x1F\x7F]) |
[:digit:] |
digits (≡ [0-9]) |
[:graph:] |
graphical (≡ [!-~] == [A-Za-z0-9!"#$%&'()*+,\-./:;<=>?@[\\\]^_`{|}~]) |
[:lower:] |
lower case (≡ [a-z]) |
[:print:] |
printable (≡ [ -~] == [ [:graph:]]) |
[:punct:] |
punctuation (≡ [!-/:-@[-`{-~]) |
[:space:] |
whitespace (≡ [\t\n\v\f\r ]) |
[:upper:] |
upper case (≡ [A-Z]) |
[:word:] |
word characters (≡ [0-9A-Za-z_]) |
[:xdigit:] |
hex digit (≡ [0-9A-Fa-f]) |
In the following example, I'll use the [:digit:]
class to look for numbers in the text. Also, I show here how to search for an exact number of characters by adding the requested number in curly braces.
func main() { text := `The answer to life, universe and everything is 42 ." pattern := "[[:digit:]]{3}" match(pattern, text) pattern = "[[:digit:]]{2}" match(pattern, text) } Output: X [[:digit:]]{3} : The answer to life, universe and everything is 42. √ [[:digit:]]{2} : The answer to life, universe and everything is 42.
You can define your own classes too by putting characters in square brackets. For example, if you want to check if some text is a valid DNA sequence that contains only the characters ACGT
then use the ^[ACGT]*$
regex:
func main() { text := "AGGCGTTGGGAACGTT" pattern := "^[ACGT]*$" match(pattern, text) text = "Not exactly a DNA sequence" match(pattern, text) } Output: √ ^[ACGT]*$ : AGGCGTTGGGAACGTT X ^[ACGT]*$ : Not exactly a DNA sequence
Using Alternatives
In some cases, there are multiple viable alternatives. Matching HTTP URLs may be characterized by a protocol schema, which is either http://
or https://
. The pipe character |
lets you choose between alternatives. Here is a regex that will sort them out: (http)|(https)://\w+\.\w{2,}
. It translates to a string that starts with http://
or https://
followed by at least one word character followed by a dot followed by at least two word characters.
func main() { pattern := `(http)|(https)://\w+\.\w{2,}` match(pattern, "http://tutsplus.com") match(pattern, "https://tutsplus.com") match(pattern, "htt://tutsplus.com") } Output: √ (http)|(https)://\w+\.\w{2,} : http://tutsplus.com √ (http)|(https)://\w+\.\w{2,} : https://tutsplus.com X (http)|(https)://\w+\.\w{2,} : htt://tutsplus.com
Conclusion
In this part of the tutorial, we covered a lot of ground and learned a lot about regular expressions, with hands-on examples using the Golang regexp library. We focused on pure matching and how to express our intentions using regular expressions.
In part two, we will focus on using regular expressions to work with text, including fuzzy finding, replacements, grouping, and dealing with new lines.
Thursday, April 26, 2018
Wednesday, April 25, 2018
Tuesday, April 24, 2018
Monday, April 23, 2018
Notifications in Laravel
In this article, we're going to explore the notification system in the Laravel web framework. The notification system in Laravel allows you to send notifications to users over different channels. Today, we'll discuss how you can send notifications over the mail channel.
Basics of Notifications
During application development, you often need to notify users about different state changes. It could be either sending email notifications when the order status is changed or sending an SMS about their login activity for security purposes. In particular, we're talking about messages that are short and just provide insight into the state changes.
Laravel already provides a built-in feature that helps us achieve something similar—notifications. In fact, it makes sending notification messages to users a breeze and a fun experience!
The beauty of that approach is that it allows you to choose from different channels notifications will be sent on. Let's quickly go through the different notification channels supported by Laravel.
- Mail: The notifications will be sent in the form of email to users.
- SMS: As the name suggests, users will receive SMS notifications on their phone.
- Slack: In this case, the notifications will be sent on Slack channels.
- Database: This option allows you to store notifications in a database should you wish to build a custom UI to display it.
Among different notification channels, we'll use the mail channel in our example use-case that we're going to develop over the course of this tutorial.
In fact, it'll be a pretty simple use-case that allows users of our application to send messages to each user. When users receive a new message in their inbox, we'll notify them about this event by sending an email to them. Of course, we'll do that by using the notification feature of Laravel!
Create a Custom Notification Class
As we discussed earlier, we are going to set up an application that allows users of our application to send messages to each other. On the other hand, we'll notify users when they receive a new message from other users via email.
In this section, we'll create necessary files that are required in order to implement the use-case that we're looking for.
To start with, let's create the Message
model that holds messages sent by users to each other.
$php artisan make:model Message --migration
We also need to add a few fields like to
, from
and message
to the messages
table. So let's change the migration file before running the migrate
command.
<?php use Illuminate\Support\Facades\Schema; use Illuminate\Database\Schema\Blueprint; use Illuminate\Database\Migrations\Migration; class CreateMessagesTable extends Migration { /** * Run the migrations. * * @return void */ public function up() { Schema::create('messages', function (Blueprint $table) { $table->increments('id'); $table->integer('from', FALSE, TRUE); $table->integer('to', FALSE, TRUE); $table->text('message'); $table->timestamps(); }); } /** * Reverse the migrations. * * @return void */ public function down() { Schema::dropIfExists('messages'); } }
Now, let's run the migrate command that creates the messages table in the database.
$php artisan migrate
That should create the messages
table in the database.
Also, make sure that you have enabled the default Laravel authentication system in the first place so that features like registration and login work out of the box. If you're not sure how to do that, the Laravel documentation provides a quick insight into that.
Since each notification in Laravel is represented by a separate class, we need to create a custom notification class that will be used to notify users. Let's use the following artisan command to create a custom notification class—NewMessage.
$php artisan make:notification NewMessage
That should create the app/Notifications/NewMessage.php
class, so let's replace the contents of that file with the following contents.
<?php // app/Notifications/NewMessage.php namespace App\Notifications; use Illuminate\Bus\Queueable; use Illuminate\Notifications\Notification; use Illuminate\Contracts\Queue\ShouldQueue; use Illuminate\Notifications\Messages\MailMessage; use App\User; class NewMessage extends Notification { use Queueable; public $fromUser; /** * Create a new notification instance. * * @return void */ public function __construct(User $user) { $this->fromUser = $user; } /** * Get the notification's delivery channels. * * @param mixed $notifiable * @return array */ public function via($notifiable) { return ['mail']; } /** * Get the mail representation of the notification. * * @param mixed $notifiable * @return \Illuminate\Notifications\Messages\MailMessage */ public function toMail($notifiable) { $subject = sprintf('%s: You\'ve got a new message from %s!', config('app.name'), $this->fromUser->name); $greeting = sprintf('Hello %s!', $notifiable->name); return (new MailMessage) ->subject($subject) ->greeting($greeting) ->salutation('Yours Faithfully') ->line('The introduction to the notification.') ->action('Notification Action', url('/')) ->line('Thank you for using our application!'); } /** * Get the array representation of the notification. * * @param mixed $notifiable * @return array */ public function toArray($notifiable) { return [ // ]; } }
As we're going to use the mail channel to send notifications to users, the via
method is configured accordingly. So this is the method that allows you to configure the channel type of a notification.
Next, there's the toMail
method that allows you to configure various email parameters. In fact, the toMail
method should return the instance of \Illuminate\Notifications\Messages\MailMessage
, and that class provides useful methods that allow you to configure email parameters.
Among various methods, the line
method allows you to add a single line in a message. On the other hand, there's the action
method that allows you to add a call-to-action button in a message.
In this way, you could format a message that will be sent to users. So that's how you're supposed to configure the notification class while you're using the mail channel to send notifications.
At the end, you need to make sure that you implement the necessary methods according to the channel type configured in the via
method. For example, if you're using the database channel that stores notifications in a database, you don't need to configure the toMail
method; instead, you should implement the toArray
method, which formats the data that needs to be stored in a database.
How to Send Notifications
In the previous section, we created a notification class that's ready to send notifications. In this section, we'll create files that demonstrate how you could actually send notifications using the NewMessage
notification class.
Let's create a controller file at app/Http/Controllers/NotificationController.php
with the following contents.
<?php namespace App\Http\Controllers; use App\Http\Controllers\Controller; use App\Message; use App\User; use App\Notifications\NewMessage; use Illuminate\Support\Facades\Notification; class NotificationController extends Controller { public function __construct() { $this->middleware('auth'); } public function index() { // user 2 sends a message to user 1 $message = new Message; $message->setAttribute('from', 2); $message->setAttribute('to', 1); $message->setAttribute('message', 'Demo message from user 2 to user 1.'); $message->save(); $fromUser = User::find(2); $toUser = User::find(1); // send notification using the "user" model, when the user receives new message $toUser->notify(new NewMessage($fromUser)); // send notification using the "Notification" facade Notification::send($toUser, new NewMessage($fromUser)); } }
Of course, you need to add an associated route in the routes/web.php
file.
Route::get('notify/index', 'NotificationController@index');
There are two ways Laravel allows you to send notifications: by using either the notifiable entity or the Notification facade.
If the entity model class utilizes the Illuminate\Notifications\Notifiable
trait, then you could call the notify
method on that model. The App\User
class implements the Notifiable
trait and thus it becomes the notifiable entity. On the other hand, you could also use the Illuminate\Support\Facades\Notification
Facade to send notifications to users.
Let's go through the index
method of the controller.
In our case, we're going to notify users when they receive a new message. So we've tried to mimic that behavior in the index
method in the first place.
Next, we've notified the recipient user about a new message using the notify
method on the $toUser
object, as it's the notifiable entity.
$toUser->notify(new NewMessage($fromUser));
You may have noticed that we also pass the $fromUser
object in the first argument of the __construct
method, since we want to include the from username in a message.
On the other hand, if you want to mimic it using the Notification
facade, it's pretty easy to do so using the following snippet.
Notification::send($toUser, new NewMessage($fromUser));
As you can see, we've used the send
method of the Notification facade to send a notification to a user.
Go ahead and open the URL http://your-laravel-site-domain/notify/index in your browser. If you're not logged in yet, you'll be redirected to the login screen. Once you're logged in, you should receive a notification email at the email address that's attached with the user 1
.
You may be wondering how the notification system detects the to
address when we haven't configured it anywhere yet. In that case, the notification system tries to find the email
property in the notifiable object. And the App\User
object class already has that property as we're using the default Laravel authentication system.
However, if you would like to override this behavior and you want to use a different property other than email, you just need to define the following method in your notification class.
public function routeNotificationForMail() { return $this->email_address; }
Now, the notification system should look for the email_address
property instead of the email
property to fetch the to
address.
And that's how to use the notification system in Laravel. That brings us to the end of this article as well!
Conclusion
What we've gone through today is one of the useful, yet least discussed, features in Laravel—notifications. It allows you to send notifications to users over different channels.
After a quick introduction, we implemented a real-world example that demonstrated how to send notifications over the mail channel. In fact, it's really handy in the case of sending short messages about state changes in your application.
For those of you who are either just getting started with Laravel or looking to expand your knowledge, site, or application with extensions, we have a variety of things you can study in Envato Market.
Should you have any queries or suggestions, don't hesitate to post them using the feed below!
Sunday, April 22, 2018
Saturday, April 21, 2018
Friday, April 20, 2018
Introduction to the Stimulus Framework
There are lots of JavaScript frameworks out there. Sometimes I even start to think that I'm the only one who has not yet created a framework. Some solutions, like Angular, are big and complex, whereas some, like Backbone (which is more a library than a framework), are quite simple and only provide a handful of tools to speed up the development process.
In today's article I would like to present you a brand new framework called Stimulus. It was created by a Basecamp team led by David Heinemeier Hansson, a popular developer who was the father of Ruby on Rails.
Stimulus is a small framework that was never intended to grow into something big. It has its very own philosophy and attitude towards front-end development, which some programmers might like or dislike. Stimulus is young, but version 1 has already been released so it should be safe to use in production. I've played with this framework quite a bit and really liked its simplicity and elegance. Hopefully, you will enjoy it too!
In this post we'll discuss the basics of Stimulus while creating a single-page application with asynchronous data loading, events, state persistence, and other common things.
The source code can be found on GitHub.
Introduction to Stimulus
Stimulus was created by developers at Basecamp. Instead of creating single-page JavaScript applications, they decided to choose a majestic monolith powered by Turbolinks and some JavaScript. This JavaScript code evolved into a small and modest framework which does not require you to spend hours and hours learning all its concepts and caveats.
Stimulus is mostly meant to attach itself to existing DOM elements and work with them in some way. It is possible, however, to dynamically render the contents as well. All in all, this framework is quite different from other popular solutions as, for example, it persists state in HTML, not in JavaScript objects. Some developers may find it inconvenient, but do give Stimulus a chance, as it really may surprise you.
The framework has only three main concepts that you should remember, which are:
- Controllers: JS classes with some methods and callbacks that attach themselves to the DOM. The attachment happens when a
data-controller
"magic" attribute appears on the page. The documentation explains that this attribute is a bridge between HTML and JavaScript, just like classes serve as bridges between HTML and CSS. One controller can be attached to multiple elements, and one element may be powered up by multiple controllers. - Actions: methods to be called on specific events. They are defined in special
data-action
attributes. - Targets: important elements that can be easily accessed and manipulated. They are specified with the help of
data-target
attributes.
As you can see, the attributes listed above allow you to separate content from behaviour logic in a very simple and natural way. Later in this article, we will see all these concepts in action and notice how easy it is to read an HTML document and understand what's going on.
Bootstrapping a Stimulus Application
Stimulus can be easily installed as an NPM package or loaded directly via the script
tag as explained in the docs. Also note that by default this framework integrates with the Webpack asset manager, which supports goodies like controller autoloading. You are free to use any other build system, but in this case some more work will be needed.
The quickest way to get started with Stimulus is by utilizing this starter project that has Express web server and Babel already hooked up. It also depends on Yarn, so be sure to install it. To clone the project and install all its dependencies, run:
git clone https://github.com/stimulusjs/stimulus-starter.git cd stimulus-starter yarn install
If you'd prefer not to install anything locally, you may remix this project on Glitch and do all the coding right in your browser.
Great—we are all set and can proceed to the next section!
Some Markup
Suppose we are creating a small single-page application that presents a list of employees and loads information like their name, photo, position, salary, birthdate, etc.
Let's start with the list of employees. All the markup that we are going to write should be placed inside the public/index.html
file, which already has some very minimal HTML. For now, we will hard-code all our employees in the following way:
<h1>Our employees</h1> <div> <ul> <li><a href="#">John Doe</a></li> <li><a href="#">Alice Smith</a></li> <li><a href="#">Will Brown</a></li> <li><a href="#">Ann Grey</a></li> </ul> </div>
Nice! Now let's add a dash of Stimulus magic.
Creating a Controller
As the official documentation explains, the main purpose of Stimulus is to connect JavaScript objects (called controllers) to the DOM elements. The controllers will then bring the page to life. As a convention, controllers' names should end with a _controller
postfix (which should be very familiar to Rails developers).
There is a directory for controllers already available called src/controllers
. Inside, you will find a hello_controller.js
file that defines an empty class:
import { Controller } from "stimulus" export default class extends Controller { }
Let's rename this file to employees_controller.js
. We don't need to specifically require it because controllers are loaded automatically thanks to the following lines of code in the src/index.js
file:
const application = Application.start() const context = require.context("./controllers", true, /\.js$/) application.load(definitionsFromContext(context))
The next step is to connect our controller to the DOM. In order to do this, set a data-controller
attribute and assign it an identifier (which is employees
in our case):
<div data-controller="employees"> <ul> <!-- your list --> </ul> </div>
That's it! The controller is now attached to the DOM.
Lifecycle Callbacks
One important thing to know about controllers is that they have three lifecycle callbacks that get fired on specific conditions:
initialize
: this callback happens only once, when the controller is instantiated.connect
: fires whenever we connect the controller to the DOM element. Since one controller may be connected to multiple elements on the page, this callback may run multiple times.disconnect
: as you've probably guessed, this callback runs whenever the controller disconnects from the DOM element.
Nothing complex, right? Let's take advantage of the initialize()
and connect()
callbacks to make sure our controller actually works:
// src/controllers/employees_controller.js export default class extends Controller { initialize() { console.log('Initialized') console.log(this) } connect() { console.log('Connected') console.log(this) } }
Next, start the server by running:
yarn start
Navigate to http://localhost:9000
. Open your browser's console and make sure both messages are displayed. It means that everything is working as expected!
Adding Events
The next core Stimulus concept is events. Events are used to respond to various user actions on the page: clicking, hovering, focusing, etc. Stimulus does not try to reinvent a bicycle, and its event system is based on generic JS events.
For instance, let's bind a click event to our employees. Whenever this event happens, I would like to call the as yet non-existent choose()
method of the employees_controller
:
<ul> <li><a href="#" data-action="click->employees#choose">John Doe</a></li> <li><a href="#" data-action="click->employees#choose">Alice Smith</a></li> <li><a href="#" data-action="click->employees#choose">Will Brown</a></li> <li><a href="#" data-action="click->employees#choose">Ann Grey</a></li> </ul>
Probably, you can understand what's going on here by yourself.
data-action
is the special attribute that binds an event to the element and explains what action should be called.click
, of course, is the event's name.employees
is the identifier of our controller.choose
is the name of the method that we'd like to call.
Since click
is the most common event, it can be safely omitted:
<li><a href="#" data-action="employees#choose">John Doe</a></li>
In this case, click
will be used implicitly.
Next, let's code the choose()
method. I don't want the default action to happen (which is, obviously, opening a new page specified in the href
attribute), so let's prevent it:
// src/controllers/employees_controller.js // callbacks here... choose(e) { e.preventDefault() console.log(this) console.log(e) }
e
is the special event object that contains full information about the triggered event. Note, by the way, that this
returns the controller itself, not an individual link! In order to gain access to the element that acts as the event's target, use e.target
.
Reload the page, click on a list item, and observe the result!
Working With the State
Now that we have bound a click event handler to the employees, I'd like to store the currently chosen person. Why? Having stored this info, we can prevent the same employee from being selected the second time. This will later allow us to avoid loading the same information multiple times as well.
Stimulus instructs us to persist state in the Data API, which seems quite reasonable. First of all, let's provide some arbitrary ids for each employee using the data-id
attribute:
<ul> <li><a href="#" data-id="1" data-action="employees#choose">John Doe</a></li> <li><a href="#" data-id="2" data-action="click->employees#choose">Alice Smith</a></li> <li><a href="#" data-id="3" data-action="click->employees#choose">Will Brown</a></li> <li><a href="#" data-id="4" data-action="click->employees#choose">Ann Grey</a></li> </ul>
Next, we need to fetch the id and persist it. Using the Data API is very common with Stimulus, so a special this.data
object is provided for each controller. With its help, we can run the following methods:
this.data.get('name')
: get the value by its attribute.this.data.set('name', value)
: set the value under some attribute.this.data.has('name')
: check if the attribute exists (returns a boolean value).
Unfortunately, these shortcuts are not available for the targets of the click events, so we must stick with getAttribute()
in their case:
// src/controllers/employees_controller.js choose(e) { e.preventDefault() this.data.set("current-employee", e.target.getAttribute('data-id')) }
But we can do even better by creating a getter and a setter for the currentEmployee
:
// src/controllers/employees_controller.js get currentEmployee() { return this.data.get("current-employee") } set currentEmployee(id) { if (this.currentEmployee !== id) { this.data.set("current-employee", id) } }
Notice how we are using the this.currentEmployee
getter and making sure that the provided id is not the same as the already stored one.
Now you may rewrite the choose()
method in the following way:
// src/controllers/employees_controller.js choose(e) { e.preventDefault() this.currentEmployee = e.target.getAttribute('data-id') }
Reload the page to make sure that everything still works. You won't notice any visual changes yet, but with the help of the Inspector tool you'll notice that the ul
has the data-employees-current-employee
attribute with a value that changes as you click on the links. The employees
part in the attribute's name is the controller's identifier and is being added automatically.
Now let's move on and highlight the currently chosen employee.
Using Targets
When an employee is selected, I would like to assign the corresponding element with a .chosen
class. Of course, we might have solved this task by using some JS selector functions, but Stimulus provides a neater solution.
Meet targets, which allow you to mark one or more important elements on the page. These elements can then be easily accessed and manipulated as needed. In order to create a target, add a data-target
attribute with the value of {controller}.{target_name}
(which is called a target descriptor):
<ul data-controller="employees"> <li><a href="#" data-target="employees.employee" data-id="1" data-action="employees#choose">John Doe</a></li> <li><a href="#" data-target="employees.employee" data-id="2" data-action="click->employees#choose">Alice Smith</a></li> <li><a href="#" data-target="employees.employee" data-id="3" data-action="click->employees#choose">Will Brown</a></li> <li><a href="#" data-target="employees.employee" data-id="4" data-action="click->employees#choose">Ann Grey</a></li> </ul>
Now let Stimulus know about these new targets by defining a new static value:
// src/controllers/employees_controller.js export default class extends Controller { static targets = [ "employee" ] // ... }
How do we access the targets now? It's as simple as saying this.employeeTarget
(to get the first element) or this.employeeTargets
(to get all the elements):
// src/controllers/employees_controller.js choose(e) { e.preventDefault() this.currentEmployee = e.target.getAttribute('data-id') console.log(this.employeeTargets) console.log(this.employeeTarget) }
Great! How can these targets help us now? Well, we can use them to add and remove CSS classes with ease based on some criteria:
// src/controllers/employees_controller.js choose(e) { e.preventDefault() this.currentEmployee = e.target.getAttribute('data-id') this.employeeTargets.forEach((el, i) => { el.classList.toggle("chosen", this.currentEmployee === el.getAttribute("data-id")) }) }
The idea is simple: we iterate over an array of targets and for each target compare its data-id
to the one stored under this.currentEmployee
. If it matches, the element is assigned the .chosen
class. Otherwise, this class is removed. You may also extract the if (this.currentEmployee !== id) {
condition from the setter and use it in the chosen()
method instead:
// src/controllers/employees_controller.js choose(e) { e.preventDefault() const id = e.target.getAttribute('data-id') if (this.currentEmployee !== id) { // <--- this.currentEmployee = id this.employeeTargets.forEach((el, i) => { el.classList.toggle("chosen", id === el.getAttribute("data-id")) }) } }
Looking nice! Lastly, we'll provide some very simple styling for the .chosen
class inside the public/main.css
:
.chosen { font-weight: bold; text-decoration: none; cursor: default; }
Reload the page once again, click on a person, and make sure that person is being highlighted properly.
Loading Data Asynchronously
Our next task is to load information about the chosen employee. In a real-world application, you would have to set up a hosting provider, a back-end powered by something like Django or Rails, and an API endpoint that responds with JSON containing all the necessary data. But we are going to make things a bit simpler and concentrate on the client side only. Create an employees
directory under the public
folder. Next, add four files containing data for individual employees:
1.json
{ "name": "John Doe", "gender": "male", "age": "40", "position": "CEO", "salary": "$120.000/year", "image": "https://burst.shopifycdn.com/photos/couple-in-love-at-sunset_373x.jpg" }
2.json
{ "name": "Alice Smith", "gender": "female", "age": "32", "position": "CTO", "salary": "$100.000/year", "image": "https://burst.shopifycdn.com/photos/woman-listening-at-team-meeting_373x.jpg" }
3.json
{ "name": "Will Brown", "gender": "male", "age": "30", "position": "Tech Lead", "salary": "$80.000/year", "image": "https://burst.shopifycdn.com/photos/casual-urban-menswear_373x.jpg" }
4.json
{ "name": "Ann Grey", "gender": "female", "age": "25", "position": "Junior Dev", "salary": "$20.000/year", "image": "https://burst.shopifycdn.com/photos/woman-using-tablet_373x.jpg" }
All photos were taken from the free stock photography by Shopify called Burst.
Our data is ready and waiting to be loaded! In order to do this, we'll code a separate loadInfoFor()
method:
// src/controllers/employees_controller.js loadInfoFor(employee_id) { fetch(`employees/${employee_id}.json`) .then(response => response.text()) .then(json => { this.displayInfo(json) }) }
This method accepts an employee's id and sends an asynchronous fetch request to the given URI. There are also two promises: one to fetch the body and another one to display the loaded info (we'll add the corresponding method in a moment).
Utilize this new method inside choose()
:
// src/controllers/employees_controller.js choose(e) { e.preventDefault() const id = e.target.getAttribute('data-id') if (this.currentEmployee !== id) { this.loadInfoFor(id) // ... } }
Before coding the displayInfo()
method, we need an element to actually render the data to. Why don't we take advantage of targets once again?
<!-- public/index.html --> <div data-controller="employees"> <div data-target="employees.info"></div> <ul> <!-- ... --> </ul> </div>
Define the target:
// src/controllers/employees_controller.js export default class extends Controller { static targets = [ "employee", "info" ] // ... }
And now utilize it to display all the info:
// src/controllers/employees_controller.js displayInfo(raw_json) { const info = JSON.parse(raw_json) const html = `<ul><li>Name: ${info.name}</li><li>Gender: ${info.gender}</li><li>Age: ${info.age}</li><li>Position: ${info.position}</li><li>Salary: ${info.salary}</li><li><img src="${info.image}"></li></ul>` this.infoTarget.innerHTML = html }
Of course, you are free to employ a templating engine like Handlebars, but for this simple case that would probably be overkill.
Now reload the page and choose one of the employees. His bio and image should be loaded nearly instantly, which means our app is working properly!
Dynamic List of Employees
Using the approach described above, we can go even further and load the list of employees on the fly rather than hard-coding it.
Prepare the data inside the public/employees.json
file:
[ { "id": "1", "name": "John Doe" }, { "id": "2", "name": "Alice Smith" }, { "id": "3", "name": "Will Brown" }, { "id": "4", "name": "Ann Grey" } ]
Now tweak the public/index.html
file by removing the hard-coded list and adding a data-employees-url
attribute (note that we must provide the controller's name, otherwise the Data API won't work):
<div data-controller="employees" data-employees-url="/employees.json"> <div data-target="employees.info"></div> </div>
As soon as controller is attached to the DOM, it should send a fetch request to build a list of employees. It means that the connect()
callback is the perfect place to do this:
// src/controllers/employees_controller.js connect() { this.loadFrom(this.data.get('url'), this.displayEmployees) }
I propose we create a more generic loadFrom()
method that accepts a URL to load data from and a callback to actually render this data:
// src/controllers/employees_controller.js loadFrom(url, callback) { fetch(url) .then(response => response.text()) .then(json => { callback.call( this, JSON.parse(json) ) }) }
Tweak the choose()
method to take advantage of the loadFrom()
:
// src/controllers/employees_controller.js choose(e) { e.preventDefault() const id = e.target.getAttribute('data-id') if (this.currentEmployee !== id) { this.loadFrom(`employees/${id}.json`, this.displayInfo) // <--- this.currentEmployee = id this.employeeTargets.forEach((el, i) => { el.classList.toggle("chosen", id === el.getAttribute("data-id")) }) } }
displayInfo()
can be simplified as well, since JSON is now being parsed right inside the loadFrom()
:
// src/controllers/employees_controller.js displayInfo(info) { const html = `<ul><li>Name: ${info.name}</li><li>Gender: ${info.gender}</li><li>Age: ${info.age}</li><li>Position: ${info.position}</li><li>Salary: ${info.salary}</li><li><img src="${info.image}"></li></ul>` this.infoTarget.innerHTML = html }
Remove loadInfoFor()
and code the displayEmployees()
method:
// src/controllers/employees_controller.js displayEmployees(employees) { let html = "<ul>" employees.forEach((el) => { html += `<li><a href="#" data-target="employees.employee" data-id="${el.id}" data-action="employees#choose">${el.name}</a></li>` }) html += "</ul>" this.element.innerHTML += html }
That's it! We are now dynamically rendering our list of employees based on the data returned by the server.
Conclusion
In this article we have covered a modest JavaScript framework called Stimulus. We have seen how to create a new application, add a controller with a bunch of callbacks and actions, and introduce events and actions. Also, we've done some asynchronous data loading with the help of fetch requests.
All in all, that's it for the basics of Stimulus—it really does not expect you to have some arcane knowledge in order to craft web applications. Of course, the framework will probably have some new features in future, but the developers are not planning to turn it into a huge monster with hundreds of tools.
If you'd like to find more examples of using Stimulus, you may also check out this tiny handbook. And if you’re looking for additional JavaScript resources to study or to use in your work, check out what we have available in the Envato Market.
Did you like Stimulus? Would you be interested in trying to create a real-world application powered by this framework? Share your thoughts in the comments!
As always, I thank you for staying with me and until the next time.
Thursday, April 19, 2018
How to Set Up a Full-Text Search Using Scout in Laravel
Full-text search is crucial for allowing users to navigate content-rich websites. In this post, I'll show you how to implement full-text search for a Laravel app. In fact, we'll use the Laravel Scout library, which makes implementation of full-text search easy and fun.
What exactly is the Laravel Scout? The official documentation sums it up like this:
Laravel Scout provides a simple, driver-based solution for adding full-text search to your Eloquent models. Using model observers, Scout will automatically keep your search indexes in sync with your Eloquent records.
Basically, Laravel Scout is a library that manages manipulation of the index whenever there's a change in the model data. The place where the data will be indexed depends on the driver that you've configured with the Scout library.
As of now, the Scout library supports Algolia, a cloud-based search engine API, and that's what we'll use in this article to demonstrate the full-text search implementation.
We'll start by installing the Scout and Algolia server libraries, and as we move on we'll go through a real-world example to demonstrate how you could index and search your data.
Server Configurations
In this section, we're going to install the dependencies that are required in order to make the Scout library work with Laravel. After installation, we'll need to go through quite a bit of configuration so that Laravel can detect the Scout library.
Let's go ahead and install the Scout library using Composer.
$composer require laravel/scout
That's pretty much it as far as the Scout library installation is concerned. Now that we've installed the Scout library, let's make sure that Laravel knows about it.
Working with Laravel, you're probably aware of the concept of a service provider, which allows you to configure services in your application. Thus, whenever you want to enable a new service in your Laravel application, you just need to add an associated service provider entry in the config/app.php
.
If you're not familiar with Laravel service providers yet, I would strongly recommend that you do yourself a favor and go through this introductory article that explains the basics of service providers in Laravel.
In our case, we just need to add the ScoutServiceProvider
provider to the list of service providers in config/app.php
, as shown in the following snippet.
... ... 'providers' => [ /* * Laravel Framework Service Providers... */ Illuminate\Auth\AuthServiceProvider::class, Illuminate\Broadcasting\BroadcastServiceProvider::class, Illuminate\Bus\BusServiceProvider::class, Illuminate\Cache\CacheServiceProvider::class, Illuminate\Foundation\Providers\ConsoleSupportServiceProvider::class, Illuminate\Cookie\CookieServiceProvider::class, Illuminate\Database\DatabaseServiceProvider::class, Illuminate\Encryption\EncryptionServiceProvider::class, Illuminate\Filesystem\FilesystemServiceProvider::class, Illuminate\Foundation\Providers\FoundationServiceProvider::class, Illuminate\Hashing\HashServiceProvider::class, Illuminate\Mail\MailServiceProvider::class, Illuminate\Notifications\NotificationServiceProvider::class, Illuminate\Pagination\PaginationServiceProvider::class, Illuminate\Pipeline\PipelineServiceProvider::class, Illuminate\Queue\QueueServiceProvider::class, Illuminate\Redis\RedisServiceProvider::class, Illuminate\Auth\Passwords\PasswordResetServiceProvider::class, Illuminate\Session\SessionServiceProvider::class, Illuminate\Translation\TranslationServiceProvider::class, Illuminate\Validation\ValidationServiceProvider::class, Illuminate\View\ViewServiceProvider::class, /* * Package Service Providers... */ Laravel\Tinker\TinkerServiceProvider::class, /* * Application Service Providers... */ App\Providers\AppServiceProvider::class, App\Providers\AuthServiceProvider::class, App\Providers\BroadcastServiceProvider::class, App\Providers\EventServiceProvider::class, App\Providers\RouteServiceProvider::class, Laravel\Scout\ScoutServiceProvider::class, ], ... ...
Now, Laravel is aware of the ScoutServiceProvider
service provider. The Scout library comes with a configuration file that allows us to set API credentials.
Let's go ahead and publish the assets provided by the Scout library using the following command.
$ php artisan vendor:publish --provider="Laravel\Scout\ScoutServiceProvider" Copied File [/vendor/laravel/scout/config/scout.php] To [/config/scout.php] Publishing complete.
As you can see, it has copied the vendor/laravel/scout/config/scout.php
file to config/scout.php
.
Next, go ahead and create an account with Algolia as we'll need API credentials in the first place. Once you have the API information, let's go ahead and configure the necessary settings in the config/scout.php
file, as shown in the following snippet.
<?php return [ /* |-------------------------------------------------------------------------- | Default Search Engine |-------------------------------------------------------------------------- | | This option controls the default search connection that gets used while | using Laravel Scout. This connection is used when syncing all models | to the search service. You should adjust this based on your needs. | | Supported: "algolia", "null" | */ 'driver' => env('SCOUT_DRIVER', 'algolia'), /* |-------------------------------------------------------------------------- | Index Prefix |-------------------------------------------------------------------------- | | Here you may specify a prefix that will be applied to all search index | names used by Scout. This prefix may be useful if you have multiple | "tenants" or applications sharing the same search infrastructure. | */ 'prefix' => env('SCOUT_PREFIX', ''), /* |-------------------------------------------------------------------------- | Queue Data Syncing |-------------------------------------------------------------------------- | | This option allows you to control if the operations that sync your data | with your search engines are queued. When this is set to "true" then | all automatic data syncing will get queued for better performance. | */ 'queue' => env('SCOUT_QUEUE', false), /* |-------------------------------------------------------------------------- | Chunk Sizes |-------------------------------------------------------------------------- | | These options allow you to control the maximum chunk size when you are | mass importing data into the search engine. This allows you to fine | tune each of these chunk sizes based on the power of the servers. | */ 'chunk' => [ 'searchable' => 500, 'unsearchable' => 500, ], /* |-------------------------------------------------------------------------- | Soft Deletes |-------------------------------------------------------------------------- | | This option allows you to control whether to keep soft deleted records in | the search indexes. Maintaining soft deleted records can be useful | if your application still needs to search for the records later. | */ 'soft_delete' => false, /* |-------------------------------------------------------------------------- | Algolia Configuration |-------------------------------------------------------------------------- | | Here you may configure your Algolia settings. Algolia is a cloud hosted | search engine which works great with Scout out of the box. Just plug | in your application ID and admin API key to get started searching. | */ 'algolia' => [ 'id' => env('ALGOLIA_APP_ID', 'STQK4DEGMA'), 'secret' => env('ALGOLIA_SECRET', '6ef572194f70201ed7ad102cc9f90e05'), ], ];
Note that we've set the value of SCOUT_DRIVER
to algolia
driver. Thus, it's required that you configure the necessary settings for the Algolia driver at the end of the file. Basically, you just need to set the id
and secret
that you've got from the Algolia account.
As you can see, we're fetching values from environment variables. So let's make sure that we set the following variables in the .env
file properly.
... ... ALGOLIA_APP_ID=STQK4DEGMA ALGOLIA_SECRET=6ef572194f70201ed7ad102cc9f90e05 ... ...
Finally, we need to install the Algolia PHP SDK, which will be used to interact with the Algolia using APIs. Let's install it using the composer as shown in the following snippet.
$composer require algolia/algoliasearch-client-php
And with that, we've installed all the dependencies that are necessary in order to post and index data to the Algolia service.
Make Models Indexable and Searchable
In the previous section, we did all the hard work to set up the Scout and Algolia libraries so that we could index and search data using the Algolia search service.
In this section, we'll go through an example that demonstrates how you could index the existing data and retrieve search results from Algolia. I assume that you have a default Post
model in your application that we'll use in our example.
The first thing that we'll need to do is to add the Laravel\Scout\Searchable
trait to the Post
model. That makes the Post
model searchable; Laravel synchronizes post records with the Algolia index every time the post record is added, updated, or deleted.
<?php namespace App; use Illuminate\Database\Eloquent\Model; use Laravel\Scout\Searchable; class Post extends Model { use Searchable; /** * The attributes that should be mutated to dates. * * @var array */ protected $dates = [ 'created_at', 'updated_at' ]; }
With that, the Post
model is search-friendly!
Next, we would like to configure the fields that should get indexed in the first place. Of course, you don't want to index all the fields of your model in Algolia to keep it effective and lightweight. In fact, more often than not, you won't need it.
You can add the toSearchableArray
in the model class to configure the fields that'll be indexed.
/** * Get the indexable data array for the model. * * @return array */ public function toSearchableArray() { $array = $this->toArray(); return array('id' => $array['id'],'name' => $array['name']); }
Now, we're ready to import and index existing Post
records into Algolia. In fact, the Scout library makes this easy by providing the following artisan command.
$php artisan scout:import "App\Post"
That should import all the records of the Post
model in a single go! They are indexed as soon as they're imported, so we're ready to query records already. Go ahead and explore the Algolia dashboard to see the imported records and other utilities.
How It Works Altogether
In this section, we'll create an example that demonstrates how to perform search and CRUD operations that are synced in real time with the Algolia index.
Go ahead and create the app/Http/Controllers/SearchController.php
file with the following contents.
<?php namespace App\Http\Controllers; use App\Http\Controllers\Controller; use App\Post; class SearchController extends Controller { public function query() { // queries to Algolia search index and returns matched records as Eloquent Models $posts = Post::search('title')->get(); // do the usual stuff here foreach ($posts as $post) { // ... } } public function add() { // this post should be indexed at Algolia right away! $post = new Post; $post->setAttribute('name', 'Another Post'); $post->setAttribute('user_id', '1'); $post->save(); } public function delete() { // this post should be removed from the index at Algolia right away! $post = Post::find(1); $post->delete(); } }
Of course, we need to add the associated routes as well.
Route::get('search/query', 'SearchController@query'); Route::get('search/add', 'SearchController@add'); Route::get('search/delete', 'SearchController@delete');
Let's go through the query
method to see how to perform a search in Algolia.
public function query() { // queries to Algolia search index and returns matched records as Eloquent Models $posts = Post::search('title')->get(); // do the usual stuff here foreach ($posts as $post) { // ... } }
Recall that we made the Post
model searchable by adding the Searchable
trait. Thus, the Post
model can use the search
method to retrieve records from the Algolia index. In the above example, we're trying to fetch records that match the title
keyword.
Next, there's the add
method that imitates the workflow of adding a new post record.
public function add() { // this post should be indexed at Algolia right away! $post = new Post; $post->setAttribute('name', 'Another Post'); $post->setAttribute('user_id', '1'); $post->save(); }
There's nothing fancy in the above code; it just creates a new post record using the Post
model. But the Post
model implements the Searchable
trait, so Laravel does some extra work this time around by indexing the newly created record in Algolia. So as you can see, the indexing is done in real time.
Finally, there's the delete
method. Let's go through it as well.
public function delete() { // this post should be removed from the index at Algolia right away! $post = Post::find(1); $post->delete(); }
As you would have expected, the record is deleted right away from the Algolia index as soon as it's deleted from the database.
Basically, there's no extra effort required from your side if you want to make existing models searchable. Everything is handled by the Scout library using model observers.
And that also brings us to the end of this article!
Conclusion
Today, we discussed how you can implement full-text search in Laravel using the Laravel Scout library. In the process, we went through the necessary installations and a real-world example to demonstrate it.
Feel free to ask if you have any queries or doubts using the comment feed below!