Nimble Collaborative Iterations: A Practical and Progressive Approach to Developing Legal Technology Tools You Will Actually Use

NIMBLE COLLABORATIVE ITERATIONS: A PRACTICAL AND PROGRESSIVE APPROACH TO DEVELOPING LEGAL TECHNOLOGY TOOLS YOU WILL ACTUALLY USE

Ko Cheng De & Neil Yap^*

As ‘LegalTech’ continues being the buzzword within the legal industry, legal technology has steadily become more widely utilised by law firms across the world. In fact, some law firms have even started to implement legal-technology blended training schemes to allow trainee lawyers the development of their skills on both the legal and the technology front. Law firms have also become more receptive to the idea of using legal technology to streamline traditional work processes.

Through the prototyping of a ‘legal decisions parser’ that uses simple programming and natural language processing techniques to extract useful information from administrative decisions, this paper seeks to unravel the issues surrounding the creation and reliance on legal technology. In particular, the paper explores matters relating to the unbundling of legal services, the synergistic potential between lawyers, engineers and legal technologists, and how the legal industry can embrace legal technology in this age of disruption – or rather, in our view, the age of opportunity.

I. INTRODUCTION

The doctrine of stare decisis plays an important role in the art of lawyering. The doctrine, as explained by Lord Gardiner, the then Lord Chancellor, ‘provides at least some degree of certainty upon which individuals can rely in the conduct of their affairs’.¹ Legal minds are therefore attuned to applying this doctrine in their practice of law. However, one wonders whether such legal doctrine of precedence has its place in the business aspects of legal services – i.e. given the ‘tried and tested’ methodology of the provision of legal services, is there a need to revisit and evaluate this pre-existing and arguably workable model in the current state of affairs? The answer is a resounding yes.

In 2007, the legal industry of the United Kingdom was introduced to alternative business structures, allowing non-lawyers to own and run legal businesses, and hence, enabling external investment to be injected into legal businesses by outside investors.² Such a development was thought to be ‘of profound significance and represent a major departure from conventional legal services’,³ as these new players (including the Big 4 accountancy giants, to which alternative business structures licences have been granted) are not committed to traditional ways of working:

‘… They are not constrained by old ways of working. They are passionate about change, and they are often better business managers than most lawyers, who tend to have had little training in the actual running of commercial concerns. How different the legal world will surely be when influenced over time by… the management methodology of corporate boards, and with the backing of venture capital and private equity.’⁴

Similarly, Singapore had, since 2014, introduced alternative business structures into the legal industry.⁵ Since 2015, individuals who are not regulated legal practitioners may hold shares in a law corporation upon successful registration to be a regulated non-practitioner,⁶ thus aligning Singapore’s legal industry with that of the United Kingdom.

A second impetus for a re-evaluation of the current business of law is the rise in legal technology solutions available for use. Such use of legal technology are broadly in two manners: one, that legal technology assists legal services providers in their provision of legal services; and two, that legal technology will ‘likely usher in an era of unprecedented legal self-help and collaboration’.⁷ As of 2016, it was suggested that 30 to 50 percent of tasks carried out by junior lawyers today may be performed by legal technology solutions.⁸

Interestingly enough, notwithstanding the above and other articles,⁹ publications,¹⁰ and initiatives¹¹ that actively recommend the assimilation of legal technology into a legal practitioner’s workflow, the Singapore legal industry’s response to legal technology has regrettably been described as ”lukewarm” by the Honourable the Chief Justice Sundaresh Menon in 2018.¹² It was particularly noted by Menon CJ that a 2017 study by the Law Society of Singapore found that ‘just 9% of small and medium-sized Singapore law practices used technology-enabled productivity tools, let alone artificial intelligence software’.¹³

Against this backdrop, this paper seeks to explore and unravel the issues surrounding the creation and reliance on legal technology through the prototyping of a ‘legal decisions parser’ as an example of how legal workflows may be unbundled and streamlined through the use of simple legal technology solutions.

Part II of this paper discusses the general trend of legal technology reliance by the legal industry, and the current adaptations implemented by law firms in light of the changing business of law. Part III turns to consider how law firms can and should embrace legal technology in their business of law. Part IV sets out the rationale and methodology behind the creation of the parser. Finally, Part V of this paper considers the future of law in light of legal technology.

II. RISE OF LEGAL TECHNOLOGY

A. What is ‘legal technology’?

Before discussing the industry’s adoption of legal technology, it is worthy to note the Singapore Academy of Law’s (‘SAL’) well-defined ambit of legal technology. In its Legal Technology Vision, the SAL first considered the ambit of ‘legal services’: legal services to any entity in need of assistance with the law.¹⁴ Based on such a broad understanding of legal services, the SAL considers legal technology to be ‘technology that enables a legal services provider to better provide value to anybody involved in understanding or applying the law’.¹⁵

Two important points arise from such a definition. First, given that legal services are not specific to legal services provided by legal practitioners to clients, legal technology should similarly not be confined to solutions for legal practitioners. A straightforward example would be the creation of a solution that is suited for legal counsels within an entity, as they may generally also render legal advice.¹⁶ Another less straightforward example would be that the solutions themselves become the provider of such ‘legal services’ (within the ambit of the Legal Profession Act, by either becoming regulated under the Legal Profession Act, or through the provision of such services in a manner which does not overreach into the boundaries of regulated services),¹⁷ such as the OCBC Online Will Generator.¹⁸

This first point is defining as it brings out the dichotomy (or rather, what is often considered to be a dichotomy) of sustenance versus disruption of legal technology to the industry. A disruptive technology impacts the side supplying the legal service and thus benefits the buyers of these services, whereas a sustaining technology enhances the market or the business process.¹⁹ Evident from the above is that legal technology will act as both sustaining and disruptive forces within the legal industry.

The second key point to note is the purpose of legal technology, which is to enable legal services providers to better provide value. In this regard, while most of the focus is on specific solutions that assist a legal practitioner in his or her legal workflow (for example, a contractual review or a legal research solution), it is imperative to point out that legal technology can (and should) also take other forms to streamline a legal practitioner’s workflow in general. Examples range from internal document and knowledge management solutions, to arguably more ‘backend’ related solutions that assist in client engagement and billing processes.

B. Legal industry’s response to legal technology

A 2018 report by the Law Society of Singapore found that more than US$2.5 billion (approximately S$3.5 billion) had been spent on legal technology deals, with the Asia Pacific region being home to about 40 dedicated legal technology firms.²⁰ With regard to Singapore, although legal technology uptake has not been the most forthcoming, it was also reported that 72 percent of the decision makers in law firms held the view that they need to increase the level of technology adoption.²¹

To this end, the law firms in Singapore have started leveraging on legal technology in various different ways, whether by adopting the use of external legal technology solutions, or by using legal technology as client-facing solutions. For example, WongPartnership adopted the use of artificial intelligence technology, Luminance, to support their corporate and mergers & acquisitions practice, and implemented a subscription-based legal document automation service called ‘D-A-S-H’ or ‘Document Automation Services Help’.²² Dentons Rodyk leveraged on Contract Companion, which is a tool to assist in the reviewing and proofreading of documents.²³ Rajah & Tann Singapore launched its digital arm, Rajah & Tann Technologies, offering a suite of technology-enabled legal solutions such as electronic discovery, contract management, and its virtual law academy.²⁴ VanillaLaw LLC implemented VanillaLaw Docs, which is an in-house interactive web-based platform that allows clients to develop their own online contract template and prepare a first draft of their legal documents by inserting relevant information.²⁵

Outside of Singapore, legal technology has taken an arguably more prominent role; in addition to adopting legal technology, international law firms have been looking further to build legal technology suitable for their specific businesses. Examples include Allen & Overy’s Fuse;²⁶ Baker Mckenzie’s collaboration with Avvoka;²⁷ Clifford Chance’s Create+65²⁸ and Clifford Chance Applied Solutions Limited;²⁹ Linklaters’ nakhoda;³⁰ Norton Rose Fulbright’s The Accord Project;³¹ and Reed Smith’s TermJet,³² to name a few.

III. HOW CAN LAW FIRMS EMBRACE LEGAL TECHNOLOGY?

A. Correlation between the rising interest in legal technology and the advancements in machine learning capabilities

It is not surprising to see that the rising interest in legal technology in the past few years follows the growing engrossment with the most buzzworthy technologies of the last decade, namely, distributed ledger technologies, artificial intelligence, and quantum computing. While the first has gained notoriety in the legal community for the interesting regulatory issues surrounding digital tokens and the third has no current practical use in the legal space, the second has already found its way into solutions that big law firms are already paying big money to use.³³

Machine learning, a subset of artificial intelligence methodologies, has garnered widespread enthusiasm not only due to accomplishing impressive feats such as beating the world Go champion with a computer,³⁴ but also through achieving significant improvements over the state-of-the-art in practical tasks, such as image recognition³⁵ and text auto-completion, which are functions we take for granted in our smartphones today. While machine learning has been around since the 1950s, the feats we have seen in the recent decade really stem from the success of deep learning,³⁶ a method of machine learning based on artificial neural networks that has realised its potential in an age of abundant data and computational resources.

In order for machine learning algorithms to deal with unstructured textual data, which is the form legal documents chiefly take, the data must be converted into a suitable input format. This usually involves an approach called natural language processing (‘NLP’), a subset of artificial intelligence which overlaps heavily with machine learning. NLP commonly involves cleaning, tokenising and converting the data into a numerical form, namely a vector representation, so that a representation (sometimes known as ‘word/sentence embeddings’) can be ‘learned’ to perform tasks such as text classification or summarisation.³⁷ Towards the end of the 2010s, advances in natural language processing combined with deep learning took the spotlight in the machine learning community, with language models such as Google’s BERT³⁸ and T5³⁹ breaking records for natural language tasks such as question answering.

While we have seen legal technology applications that use machine learning techniques, such as unsupervised learning in due diligence contract review applications, why has the response of law firms to legal technology not reflected the level of success seen in the natural language processing field? First, deep learning methods are extremely data hungry.⁴⁰ ImageNet is a dataset of over 15 million labelled images. While there is no need for millions of training samples to get deep learning models to yield benefit, hundreds of samples are still required at the minimum. More importantly, these samples should be labelled properly if the samples are for a supervised task such as document categorisation. Quality-labelled data in the legal domain is scarce, and there is little evidence of law firms making concerted efforts to create quality-labelled training sets.

The best training examples that do exist in the real world for the legal domain will probably be tagged publicly available law reports, but a model trained on this would be unlikely to perform well on tasks such as contract clause classification. This ties nicely into the second potential issue, which is that it is hard to find representative training data that is optimised for legal tasks. BERT was trained with petabytes of text and required copious amounts of computational resources, but it was trained on a very general corpus. Recall that machine learning models need to take numerical representations of texts before they can ‘learn’ the meaning of words and phrases, and that the meanings ‘learnt’ inevitably come from the training text corpus. For example, the vector representing the word ‘frustration’ would be close to that of the word ‘performance’ if the representation is learned from a corpus of legal texts. On a general corpus, ‘frustration’ might not be as close to ‘performance’ as it is to ‘despair’. That being said, on either corpus, ‘frustration’ would be found close to ‘covid-19’. While the benefits of large language models can be technically adapted to more specific domains through improvements in a process called ‘transfer learning’,⁴¹ success on this front in the legal space, at least in a commercial setting, has yet to be shown. It is still early days and one can be hopeful, but fine-tuning still ties back to the earlier requirement of having a representative corpus or dataset to adapt the language model to. In the context of the legal domain, what this ‘representative corpus’ is could vary widely depending on the task, and the jury is still out for accomplishing practical legal tasks.

Recent advances in deep learning and NLP have yet to translate to similar levels of success in the legal technology field despite sharing parallel timelines. One should not necessarily assume that progress in the former necessarily means progress in the latter. Staying optimistic, that is far from saying that it is impossible for legal practitioners to tap on state-of-the-art NLP models to work for them. However, it will undoubtedly take effort to assemble appropriate data, work with NLP experts to test and fine tune their applications, and come up with proper methods and benchmarks to objectively determine their impact. For the legal community to yield such benefits, there must be more efforts to tailor, collate and collaborate to produce such datasets. However, due to the confidential nature of legal documents, the most practical hope for the legal community comes from public authorities and regulators being willing to make data available in an abundant, structured and consistently labelled way. Another way would be for more expert lawyers to work closely with NLP practitioners to figure out linguistic features that are particular to legal contexts, for instance, encoding the syntactic structure of contracts, legislation or court submissions into learnable features.

B. More than a ‘plug and play’

The current approach to legal technology has generally been that of a ‘plug and play’, where law firms introduce pre-existing legal technology solutions to deal with well-defined and broad categories of workflows, such as contractual review, due diligence process, and legal research. While these ‘plug and play’ solutions have worked well in certain cases to assist legal practitioners in their particular fields of work, some legal practitioners find it difficult to rely on these introduced solutions for various reasons. Key issues typically are that the solution is too complex, that it takes up too much time just to learn the basic functions (much less allow it to assist in complex tasks), that it simply does not fit the legal practitioner’s personal style or workflow,⁴² or that such solutions are not well-suited for local practice.⁴³

However, this paper explores a different approach to legal technology. One that aims to achieve improvements with quick and real impact on everyday legal processes and does not require the level of investment of resources of the aforementioned methods, and that any enthusiastic lawyer can start on today. This can be done by looking at legal technology as simply another tool in a lawyer’s toolbox.

Apart from traditional abilities, such as advocacy or drafting skills, applying one’s legal knowledge to improve a part of an everyday lawyer’s workflow is another form of delivering value, whether to clients, judges, prosecutors, or other individuals. We recommend taking a two-step approach when looking to implement a legal technology solution.

1. Identifying a problem within your workflow

First, start small with a well-defined problem that is encountered in day-to-day work. Articulate the problem, the improvement you would like to see and how technology might help. To do that, the problem should not be overly broad; for example, the contract review itself should not be the problem. Instead, consider specific problems during the contract review process that may be cumbersome, tedious, or simply able to be streamlined through the use of technology. The problem may be as simple as finding the following simple tasks tedious and time consuming: to check for whether a Unique Entity Number (‘UEN’) of an entity is correctly entered, and if the entity is regulated as a financial institution in Singapore (the “Example”).

As a note, we recommend against jumping on machine learning as the solution, given the difficulties explored above. However, if you do come to the conclusion that machine learning would be a great way to solve your problem, make sure you work closely with a data scientist that understands the law firm’s processes and can work closely with users of the solution to train the models appropriately, check the suitability of the data and properly evaluate the success of the models. Machine learning models are not trivial to train and deploy, at least not for the traditional staff a law firm would have, so such complexities should be factored in too.

2. Consider the available solutions

Having identified the problem, the next step would be to identify a solution. Following on from the Example, the solution is clearly one which will not require intricate technology or sophisticated IT infrastructure to deploy. In such situations, it is likely that a pre-existing ‘plug and play’ solution may not be able to effectively deal with this work-process problem,⁴⁴ or that while it may deal with the problem, it also contains other features which either may not be helpful in your practice area, or is too expensive to purchase. Our suggestion is therefore for law firms to team up with technologists experienced in this field of work to create a legal technology solution to streamline the simple problem arising from the Example. Such collaborations should allow for the creation of quick iterations of the solution for testing and troubleshooting by the legal practitioner, so as to align the legal practitioner’s expectations and requirements with the technologist’s understanding of the problem.

As with all other types of software solutions, a key success factor will be the ability of a technologist to understand the legal problem faced, and for the legal practitioner to understand ‘technical speak’. To this end, a ‘legal technologist’ (i.e. one who is “trained and experienced both in the practice of law and in the profession of systems engineering and technology management”)⁴⁵ will be a prime candidate to ensure that both fronts of technical and legal aspects are well understood during the creation process. If we may posit this further, the role of legal technologists will progressively become more important (possibly even indispensable) within law firms, given the ever-growing legal technology market.

However, the existence of legal technologists in the current market is rare. To prepare for this impending demand, the Singapore Management University has recently introduced a one-of-a-kind Computing and Law degree programme in Singapore, seeking to equip its graduates with both technological and legal skills.⁴⁶ The programme will commence in August 2020, paving way for the first wave of Singapore-trained legal technologists.

Law firms have also started to alter their training regimes in order to mould trainees into their version of legal technologists. For example, Clifford Chance introduced IGNITE as an alternative training contract scheme, allowing individuals to qualify as lawyers through a training program which has an emphasis on technology.⁴⁷ Allen & Overy also launched a new graduate scheme, which focuses on legal technology and project management.⁴⁸

In the meantime, an avenue which law firms may explore is the FLIP Programme,⁴⁹ a programme backed by the SAL. Through FLIP, law firms may be able to gain exposure to the legal technology scene in Singapore, including gaining access to resources and collaboration opportunities with FLIP and its partners. Along the same line, law firms should also be aware of Tech-celerate for Law, which is a support scheme launched by the Law Society of Singapore for the adoption of technology solutions by Singapore Law Practices.⁵⁰ This support scheme seeks to provide up to 80 percent funding support to law firms in the first year of implementation of legal technology solutions.

To this end, we have prepared a legal decisions parser as an example of how a collaborative effort between legal practitioners and technologists may turn into a simple legal technology solution that assists a particular workflow of a legal practitioner.

IV. LEGAL DECISIONS PARSER

In this instance, the problem to solve was allowing a legal practitioner within the Technology, Media, and Telecommunications (“TMT”) team to quickly digest the key points and applicability of incoming Personal Data Protection Commission (‘PDPC’) decisions. The proposed solution was to come with a simple parser that could ingest the original PDF decision, classify the document by the Personal Data Protection Act (‘PDPA’) Obligation covered, and extract further key dimensions of interest to the TMT lawyers.

For the reasons discussed in Part III above, a relatively young area of law such as data protection will not be a great area for machine learning to tackle. The scarce amount of data means that there are not enough samples to learn topics from. The PDPC makes a commendable effort in making their data open and well-labelled. Since first embarking on this project, we noticed that the PDPC recently updated its website to include labelling its decisions by PDPA Obligation type. We also noticed that their decisions are generally consistently worded and do not vary that much in structure as compared to, for example, judgments.

Before the recent PDPC website update that tagged the documents by Obligation, one of the dimensions we explored was to automatically categorise the document with the type of Obligation the decision addressed. We focused on trying to automatically detect one of the more important Obligation types, the Protection Obligation. To do this, we first downloaded a sample of 60 decisions that we determined, through reading, were about the Protection Obligation. From there, simple statistical NLP methods (without any machine learning or word embeddings) were used to analyse key vocabulary found in the corpus. This involved creating a vocabulary of phrases found in the corpus and analysing the vocabulary from three angles, with the goal of determining the key phrases that we could use to identify that a document was about the Protection Obligation.

First, we looked at the vocabulary based on the total frequency of occurrences throughout the corpus, meaning, how many times each phrase appeared in each of the documents. This gave us an overall sense of the phrases that appeared very often in documents about the Protection Obligation. Second, in order to consider phrases that might not appear as many times in one document, but appear across many documents on the Protection Obligation, we calculated which phrases had occurred in the highest percentage of the documents. In doing so, we harnessed background legal knowledge of PDPC decisions and disregarded common PDPA words such as ‘data’, ‘commissioner’, ‘section 50’ (which is just about the investigative power of the Commissioner) and ‘in breach’. These are words and phrases that we determined, given our knowledge on the subject, would not be useful in distinguishing whether a document was about the Protection Obligation or another PDPA topic. With these first two metrics, the phrases we evaluated to be useful for classifying the Protection Obligation were as follows:

Word/ Phrase	Frequency in Corpus	% of PO Documents Containing the word/ Phrase
‘reasonable security arrangements’	159	79%
’24 of the PDPA’	333	86%
‘unauthorised access’	148	78%

For the third angle, we evaluated the set of documents with the Term Frequency-Inverse Document Frequency (tf-idf) of the phrases in the corpus vocabulary. This is a composite metric that scores the word or phrase highly based on their raw frequency, but with a penalty for appearing in too many documents.⁵¹ For example, this method would thus naturally discount words like ‘PDPA’ and ‘commissioner’ that appear often, but in too many documents. Our observation is that the method is useful for exposing subtopics that could be used to further classify Protection Obligation cases based on fact patterns, such as in the phrases ‘resumes uploaded’, ‘by search engine’ and ‘external web developer’.

Beyond determining the Obligation, the further dimensions of interest we want to extract are: the entities involved; the factual elements of the case; the discussion of aggravating and mitigating factors; the direction given by the commissioner; and whether a breach was found. For many decisions, the entities can be extracted based on finding their position in the formatted party names at the top of the decision. To extract the portion discussing aggravating and mitigating factors, we look out for the terms ‘aggravating’, ‘mitigating’, ‘factors’, and ‘considerations’ in close proximity to a list pattern. To determine commissioner directions, an example pattern is that we look out for the ‘warning’ or monetary amounts in the same paragraph as the phrase, ‘Commissioner directs’. If such a direction is found, we can infer that a breach was found.

The collective effort of coding and iterating took us about a total of three days with promising results. When we are satisfied with the extraction capabilities, the plan is to have the extractor periodically run on a folder of collected PDPC decisions and writing the extraction output onto a spreadsheet that can be easily consumed by the team, a process that is currently manually done.

This is just the start and we have plans to do further iterations to improve its accuracy and coverage, to try to extract further dimensions, and to develop a simple pipeline that will be easy to use. The code repository that spells out the extraction logic and our detailed findings will be periodically updated at https://github.com/nysk92/pdpc_parsers.

V. FUTURE OF LAW

Law firms are known to operate in a model which is often called a ‘pyramidal structure’,⁵² where senior partners form the top of the pyramid while junior lawyers form the broader base of the pyramid.⁵³ However, through the utilisation of legal technology, this base of the pyramid is likely to erode, either due to the effectiveness of streamlining, or in certain other cases, complete automation of such work processes.⁵⁴ Given the foregoing, it is prudent for law firms to, in addition to considering how legal technology can assist in streamlining workflows, envision the impact legal technology will have on the existing pyramidal structure.

In the same vein, aspiring lawyers should also be aware of such potential change to the business of law, given the likelihood of a shift from the pyramidal structure which will then likely result in the reduced intake of practice trainees and junior lawyers. However, should aspiring lawyers worry? We think not.

While legal technology may generally reduce the need for junior lawyers, a further effect of legal technology integration is the rising need for legal technologists in law firms. We also believe that given the rising importance of legal technology within the legal industry, a genuine interest in legal technology, coupled with sound technical skills, will likely bode well with recruitment partners of law firms.

Finally, anxious law students should also note the key recommendations put forward by the Committee for the Professional Training of Lawyers on strengthening the professional training regime for lawyers in Singapore. These recommendations have been accepted by the Ministry of Law in 2018.⁵⁵ Hence, changes to qualifying into the legal profession are afoot. Such changes include allowing law graduates to be admitted into the Singapore Bar without having to undergo practice training, which carries the intention of allowing law graduates to pivot into the myriad of pathways that may be pursued within the legal profession.⁵⁶

VI. CONCLUDING NOTE

Coming back to revisit the purpose of Lord Gardiner’s practice statement, it should be noted that the pronouncement in 1966 had cautioned against the “rigid adherence to precedent’. As such, it amended the doctrine of stare decisis such that the House of Lords may depart from a previous decision when it appears right to do so.⁵⁷ Given the discussed potential of legal technology, the outlook of the legal industry is likely to evolve dramatically. Hence, the business of law should, too.

With that in mind, we end off with a quote from Menon CJ (who incidentally, as the managing partner of Rajah & Tann from 2009 – 2010,⁵⁸ had also chartered the path of a major Singapore law firm in its business of law):

‘The legal profession today is mired in change and the challenge we face is profound. But even as technology comes to dominate so much of our professional lives, let us not discount the power of human ingenuity and endeavour to shape the future according to our will. We have always been the masters of our own destiny and, in this moment, we have a golden opportunity to chart a new course for the entire profession. And because we can only do this together, I hope that you will all step forward and take up your places at the vanguard of change.’⁵⁹