generated from daniil-berg/boilerplate-py
	Compare commits
	
		
			8 Commits
		
	
	
		
			v0.1.7-lw
			...
			538b9cc91c
		
	
	| Author | SHA1 | Date | |
|---|---|---|---|
| 538b9cc91c | |||
| 3fb451a00e | |||
| be03097bf4 | |||
| 024e5db0d4 | |||
| bc9d2f243e | |||
| 012c8ac639 | |||
| 8fd40839ee | |||
| 36d026f433 | 
							
								
								
									
										674
									
								
								COPYING
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										674
									
								
								COPYING
									
									
									
									
									
										Normal file
									
								
							| @@ -0,0 +1,674 @@ | |||||||
|  |                     GNU GENERAL PUBLIC LICENSE | ||||||
|  |                        Version 3, 29 June 2007 | ||||||
|  |  | ||||||
|  |  Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/> | ||||||
|  |  Everyone is permitted to copy and distribute verbatim copies | ||||||
|  |  of this license document, but changing it is not allowed. | ||||||
|  |  | ||||||
|  |                             Preamble | ||||||
|  |  | ||||||
|  |   The GNU General Public License is a free, copyleft license for | ||||||
|  | software and other kinds of works. | ||||||
|  |  | ||||||
|  |   The licenses for most software and other practical works are designed | ||||||
|  | to take away your freedom to share and change the works.  By contrast, | ||||||
|  | the GNU General Public License is intended to guarantee your freedom to | ||||||
|  | share and change all versions of a program--to make sure it remains free | ||||||
|  | software for all its users.  We, the Free Software Foundation, use the | ||||||
|  | GNU General Public License for most of our software; it applies also to | ||||||
|  | any other work released this way by its authors.  You can apply it to | ||||||
|  | your programs, too. | ||||||
|  |  | ||||||
|  |   When we speak of free software, we are referring to freedom, not | ||||||
|  | price.  Our General Public Licenses are designed to make sure that you | ||||||
|  | have the freedom to distribute copies of free software (and charge for | ||||||
|  | them if you wish), that you receive source code or can get it if you | ||||||
|  | want it, that you can change the software or use pieces of it in new | ||||||
|  | free programs, and that you know you can do these things. | ||||||
|  |  | ||||||
|  |   To protect your rights, we need to prevent others from denying you | ||||||
|  | these rights or asking you to surrender the rights.  Therefore, you have | ||||||
|  | certain responsibilities if you distribute copies of the software, or if | ||||||
|  | you modify it: responsibilities to respect the freedom of others. | ||||||
|  |  | ||||||
|  |   For example, if you distribute copies of such a program, whether | ||||||
|  | gratis or for a fee, you must pass on to the recipients the same | ||||||
|  | freedoms that you received.  You must make sure that they, too, receive | ||||||
|  | or can get the source code.  And you must show them these terms so they | ||||||
|  | know their rights. | ||||||
|  |  | ||||||
|  |   Developers that use the GNU GPL protect your rights with two steps: | ||||||
|  | (1) assert copyright on the software, and (2) offer you this License | ||||||
|  | giving you legal permission to copy, distribute and/or modify it. | ||||||
|  |  | ||||||
|  |   For the developers' and authors' protection, the GPL clearly explains | ||||||
|  | that there is no warranty for this free software.  For both users' and | ||||||
|  | authors' sake, the GPL requires that modified versions be marked as | ||||||
|  | changed, so that their problems will not be attributed erroneously to | ||||||
|  | authors of previous versions. | ||||||
|  |  | ||||||
|  |   Some devices are designed to deny users access to install or run | ||||||
|  | modified versions of the software inside them, although the manufacturer | ||||||
|  | can do so.  This is fundamentally incompatible with the aim of | ||||||
|  | protecting users' freedom to change the software.  The systematic | ||||||
|  | pattern of such abuse occurs in the area of products for individuals to | ||||||
|  | use, which is precisely where it is most unacceptable.  Therefore, we | ||||||
|  | have designed this version of the GPL to prohibit the practice for those | ||||||
|  | products.  If such problems arise substantially in other domains, we | ||||||
|  | stand ready to extend this provision to those domains in future versions | ||||||
|  | of the GPL, as needed to protect the freedom of users. | ||||||
|  |  | ||||||
|  |   Finally, every program is threatened constantly by software patents. | ||||||
|  | States should not allow patents to restrict development and use of | ||||||
|  | software on general-purpose computers, but in those that do, we wish to | ||||||
|  | avoid the special danger that patents applied to a free program could | ||||||
|  | make it effectively proprietary.  To prevent this, the GPL assures that | ||||||
|  | patents cannot be used to render the program non-free. | ||||||
|  |  | ||||||
|  |   The precise terms and conditions for copying, distribution and | ||||||
|  | modification follow. | ||||||
|  |  | ||||||
|  |                        TERMS AND CONDITIONS | ||||||
|  |  | ||||||
|  |   0. Definitions. | ||||||
|  |  | ||||||
|  |   "This License" refers to version 3 of the GNU General Public License. | ||||||
|  |  | ||||||
|  |   "Copyright" also means copyright-like laws that apply to other kinds of | ||||||
|  | works, such as semiconductor masks. | ||||||
|  |  | ||||||
|  |   "The Program" refers to any copyrightable work licensed under this | ||||||
|  | License.  Each licensee is addressed as "you".  "Licensees" and | ||||||
|  | "recipients" may be individuals or organizations. | ||||||
|  |  | ||||||
|  |   To "modify" a work means to copy from or adapt all or part of the work | ||||||
|  | in a fashion requiring copyright permission, other than the making of an | ||||||
|  | exact copy.  The resulting work is called a "modified version" of the | ||||||
|  | earlier work or a work "based on" the earlier work. | ||||||
|  |  | ||||||
|  |   A "covered work" means either the unmodified Program or a work based | ||||||
|  | on the Program. | ||||||
|  |  | ||||||
|  |   To "propagate" a work means to do anything with it that, without | ||||||
|  | permission, would make you directly or secondarily liable for | ||||||
|  | infringement under applicable copyright law, except executing it on a | ||||||
|  | computer or modifying a private copy.  Propagation includes copying, | ||||||
|  | distribution (with or without modification), making available to the | ||||||
|  | public, and in some countries other activities as well. | ||||||
|  |  | ||||||
|  |   To "convey" a work means any kind of propagation that enables other | ||||||
|  | parties to make or receive copies.  Mere interaction with a user through | ||||||
|  | a computer network, with no transfer of a copy, is not conveying. | ||||||
|  |  | ||||||
|  |   An interactive user interface displays "Appropriate Legal Notices" | ||||||
|  | to the extent that it includes a convenient and prominently visible | ||||||
|  | feature that (1) displays an appropriate copyright notice, and (2) | ||||||
|  | tells the user that there is no warranty for the work (except to the | ||||||
|  | extent that warranties are provided), that licensees may convey the | ||||||
|  | work under this License, and how to view a copy of this License.  If | ||||||
|  | the interface presents a list of user commands or options, such as a | ||||||
|  | menu, a prominent item in the list meets this criterion. | ||||||
|  |  | ||||||
|  |   1. Source Code. | ||||||
|  |  | ||||||
|  |   The "source code" for a work means the preferred form of the work | ||||||
|  | for making modifications to it.  "Object code" means any non-source | ||||||
|  | form of a work. | ||||||
|  |  | ||||||
|  |   A "Standard Interface" means an interface that either is an official | ||||||
|  | standard defined by a recognized standards body, or, in the case of | ||||||
|  | interfaces specified for a particular programming language, one that | ||||||
|  | is widely used among developers working in that language. | ||||||
|  |  | ||||||
|  |   The "System Libraries" of an executable work include anything, other | ||||||
|  | than the work as a whole, that (a) is included in the normal form of | ||||||
|  | packaging a Major Component, but which is not part of that Major | ||||||
|  | Component, and (b) serves only to enable use of the work with that | ||||||
|  | Major Component, or to implement a Standard Interface for which an | ||||||
|  | implementation is available to the public in source code form.  A | ||||||
|  | "Major Component", in this context, means a major essential component | ||||||
|  | (kernel, window system, and so on) of the specific operating system | ||||||
|  | (if any) on which the executable work runs, or a compiler used to | ||||||
|  | produce the work, or an object code interpreter used to run it. | ||||||
|  |  | ||||||
|  |   The "Corresponding Source" for a work in object code form means all | ||||||
|  | the source code needed to generate, install, and (for an executable | ||||||
|  | work) run the object code and to modify the work, including scripts to | ||||||
|  | control those activities.  However, it does not include the work's | ||||||
|  | System Libraries, or general-purpose tools or generally available free | ||||||
|  | programs which are used unmodified in performing those activities but | ||||||
|  | which are not part of the work.  For example, Corresponding Source | ||||||
|  | includes interface definition files associated with source files for | ||||||
|  | the work, and the source code for shared libraries and dynamically | ||||||
|  | linked subprograms that the work is specifically designed to require, | ||||||
|  | such as by intimate data communication or control flow between those | ||||||
|  | subprograms and other parts of the work. | ||||||
|  |  | ||||||
|  |   The Corresponding Source need not include anything that users | ||||||
|  | can regenerate automatically from other parts of the Corresponding | ||||||
|  | Source. | ||||||
|  |  | ||||||
|  |   The Corresponding Source for a work in source code form is that | ||||||
|  | same work. | ||||||
|  |  | ||||||
|  |   2. Basic Permissions. | ||||||
|  |  | ||||||
|  |   All rights granted under this License are granted for the term of | ||||||
|  | copyright on the Program, and are irrevocable provided the stated | ||||||
|  | conditions are met.  This License explicitly affirms your unlimited | ||||||
|  | permission to run the unmodified Program.  The output from running a | ||||||
|  | covered work is covered by this License only if the output, given its | ||||||
|  | content, constitutes a covered work.  This License acknowledges your | ||||||
|  | rights of fair use or other equivalent, as provided by copyright law. | ||||||
|  |  | ||||||
|  |   You may make, run and propagate covered works that you do not | ||||||
|  | convey, without conditions so long as your license otherwise remains | ||||||
|  | in force.  You may convey covered works to others for the sole purpose | ||||||
|  | of having them make modifications exclusively for you, or provide you | ||||||
|  | with facilities for running those works, provided that you comply with | ||||||
|  | the terms of this License in conveying all material for which you do | ||||||
|  | not control copyright.  Those thus making or running the covered works | ||||||
|  | for you must do so exclusively on your behalf, under your direction | ||||||
|  | and control, on terms that prohibit them from making any copies of | ||||||
|  | your copyrighted material outside their relationship with you. | ||||||
|  |  | ||||||
|  |   Conveying under any other circumstances is permitted solely under | ||||||
|  | the conditions stated below.  Sublicensing is not allowed; section 10 | ||||||
|  | makes it unnecessary. | ||||||
|  |  | ||||||
|  |   3. Protecting Users' Legal Rights From Anti-Circumvention Law. | ||||||
|  |  | ||||||
|  |   No covered work shall be deemed part of an effective technological | ||||||
|  | measure under any applicable law fulfilling obligations under article | ||||||
|  | 11 of the WIPO copyright treaty adopted on 20 December 1996, or | ||||||
|  | similar laws prohibiting or restricting circumvention of such | ||||||
|  | measures. | ||||||
|  |  | ||||||
|  |   When you convey a covered work, you waive any legal power to forbid | ||||||
|  | circumvention of technological measures to the extent such circumvention | ||||||
|  | is effected by exercising rights under this License with respect to | ||||||
|  | the covered work, and you disclaim any intention to limit operation or | ||||||
|  | modification of the work as a means of enforcing, against the work's | ||||||
|  | users, your or third parties' legal rights to forbid circumvention of | ||||||
|  | technological measures. | ||||||
|  |  | ||||||
|  |   4. Conveying Verbatim Copies. | ||||||
|  |  | ||||||
|  |   You may convey verbatim copies of the Program's source code as you | ||||||
|  | receive it, in any medium, provided that you conspicuously and | ||||||
|  | appropriately publish on each copy an appropriate copyright notice; | ||||||
|  | keep intact all notices stating that this License and any | ||||||
|  | non-permissive terms added in accord with section 7 apply to the code; | ||||||
|  | keep intact all notices of the absence of any warranty; and give all | ||||||
|  | recipients a copy of this License along with the Program. | ||||||
|  |  | ||||||
|  |   You may charge any price or no price for each copy that you convey, | ||||||
|  | and you may offer support or warranty protection for a fee. | ||||||
|  |  | ||||||
|  |   5. Conveying Modified Source Versions. | ||||||
|  |  | ||||||
|  |   You may convey a work based on the Program, or the modifications to | ||||||
|  | produce it from the Program, in the form of source code under the | ||||||
|  | terms of section 4, provided that you also meet all of these conditions: | ||||||
|  |  | ||||||
|  |     a) The work must carry prominent notices stating that you modified | ||||||
|  |     it, and giving a relevant date. | ||||||
|  |  | ||||||
|  |     b) The work must carry prominent notices stating that it is | ||||||
|  |     released under this License and any conditions added under section | ||||||
|  |     7.  This requirement modifies the requirement in section 4 to | ||||||
|  |     "keep intact all notices". | ||||||
|  |  | ||||||
|  |     c) You must license the entire work, as a whole, under this | ||||||
|  |     License to anyone who comes into possession of a copy.  This | ||||||
|  |     License will therefore apply, along with any applicable section 7 | ||||||
|  |     additional terms, to the whole of the work, and all its parts, | ||||||
|  |     regardless of how they are packaged.  This License gives no | ||||||
|  |     permission to license the work in any other way, but it does not | ||||||
|  |     invalidate such permission if you have separately received it. | ||||||
|  |  | ||||||
|  |     d) If the work has interactive user interfaces, each must display | ||||||
|  |     Appropriate Legal Notices; however, if the Program has interactive | ||||||
|  |     interfaces that do not display Appropriate Legal Notices, your | ||||||
|  |     work need not make them do so. | ||||||
|  |  | ||||||
|  |   A compilation of a covered work with other separate and independent | ||||||
|  | works, which are not by their nature extensions of the covered work, | ||||||
|  | and which are not combined with it such as to form a larger program, | ||||||
|  | in or on a volume of a storage or distribution medium, is called an | ||||||
|  | "aggregate" if the compilation and its resulting copyright are not | ||||||
|  | used to limit the access or legal rights of the compilation's users | ||||||
|  | beyond what the individual works permit.  Inclusion of a covered work | ||||||
|  | in an aggregate does not cause this License to apply to the other | ||||||
|  | parts of the aggregate. | ||||||
|  |  | ||||||
|  |   6. Conveying Non-Source Forms. | ||||||
|  |  | ||||||
|  |   You may convey a covered work in object code form under the terms | ||||||
|  | of sections 4 and 5, provided that you also convey the | ||||||
|  | machine-readable Corresponding Source under the terms of this License, | ||||||
|  | in one of these ways: | ||||||
|  |  | ||||||
|  |     a) Convey the object code in, or embodied in, a physical product | ||||||
|  |     (including a physical distribution medium), accompanied by the | ||||||
|  |     Corresponding Source fixed on a durable physical medium | ||||||
|  |     customarily used for software interchange. | ||||||
|  |  | ||||||
|  |     b) Convey the object code in, or embodied in, a physical product | ||||||
|  |     (including a physical distribution medium), accompanied by a | ||||||
|  |     written offer, valid for at least three years and valid for as | ||||||
|  |     long as you offer spare parts or customer support for that product | ||||||
|  |     model, to give anyone who possesses the object code either (1) a | ||||||
|  |     copy of the Corresponding Source for all the software in the | ||||||
|  |     product that is covered by this License, on a durable physical | ||||||
|  |     medium customarily used for software interchange, for a price no | ||||||
|  |     more than your reasonable cost of physically performing this | ||||||
|  |     conveying of source, or (2) access to copy the | ||||||
|  |     Corresponding Source from a network server at no charge. | ||||||
|  |  | ||||||
|  |     c) Convey individual copies of the object code with a copy of the | ||||||
|  |     written offer to provide the Corresponding Source.  This | ||||||
|  |     alternative is allowed only occasionally and noncommercially, and | ||||||
|  |     only if you received the object code with such an offer, in accord | ||||||
|  |     with subsection 6b. | ||||||
|  |  | ||||||
|  |     d) Convey the object code by offering access from a designated | ||||||
|  |     place (gratis or for a charge), and offer equivalent access to the | ||||||
|  |     Corresponding Source in the same way through the same place at no | ||||||
|  |     further charge.  You need not require recipients to copy the | ||||||
|  |     Corresponding Source along with the object code.  If the place to | ||||||
|  |     copy the object code is a network server, the Corresponding Source | ||||||
|  |     may be on a different server (operated by you or a third party) | ||||||
|  |     that supports equivalent copying facilities, provided you maintain | ||||||
|  |     clear directions next to the object code saying where to find the | ||||||
|  |     Corresponding Source.  Regardless of what server hosts the | ||||||
|  |     Corresponding Source, you remain obligated to ensure that it is | ||||||
|  |     available for as long as needed to satisfy these requirements. | ||||||
|  |  | ||||||
|  |     e) Convey the object code using peer-to-peer transmission, provided | ||||||
|  |     you inform other peers where the object code and Corresponding | ||||||
|  |     Source of the work are being offered to the general public at no | ||||||
|  |     charge under subsection 6d. | ||||||
|  |  | ||||||
|  |   A separable portion of the object code, whose source code is excluded | ||||||
|  | from the Corresponding Source as a System Library, need not be | ||||||
|  | included in conveying the object code work. | ||||||
|  |  | ||||||
|  |   A "User Product" is either (1) a "consumer product", which means any | ||||||
|  | tangible personal property which is normally used for personal, family, | ||||||
|  | or household purposes, or (2) anything designed or sold for incorporation | ||||||
|  | into a dwelling.  In determining whether a product is a consumer product, | ||||||
|  | doubtful cases shall be resolved in favor of coverage.  For a particular | ||||||
|  | product received by a particular user, "normally used" refers to a | ||||||
|  | typical or common use of that class of product, regardless of the status | ||||||
|  | of the particular user or of the way in which the particular user | ||||||
|  | actually uses, or expects or is expected to use, the product.  A product | ||||||
|  | is a consumer product regardless of whether the product has substantial | ||||||
|  | commercial, industrial or non-consumer uses, unless such uses represent | ||||||
|  | the only significant mode of use of the product. | ||||||
|  |  | ||||||
|  |   "Installation Information" for a User Product means any methods, | ||||||
|  | procedures, authorization keys, or other information required to install | ||||||
|  | and execute modified versions of a covered work in that User Product from | ||||||
|  | a modified version of its Corresponding Source.  The information must | ||||||
|  | suffice to ensure that the continued functioning of the modified object | ||||||
|  | code is in no case prevented or interfered with solely because | ||||||
|  | modification has been made. | ||||||
|  |  | ||||||
|  |   If you convey an object code work under this section in, or with, or | ||||||
|  | specifically for use in, a User Product, and the conveying occurs as | ||||||
|  | part of a transaction in which the right of possession and use of the | ||||||
|  | User Product is transferred to the recipient in perpetuity or for a | ||||||
|  | fixed term (regardless of how the transaction is characterized), the | ||||||
|  | Corresponding Source conveyed under this section must be accompanied | ||||||
|  | by the Installation Information.  But this requirement does not apply | ||||||
|  | if neither you nor any third party retains the ability to install | ||||||
|  | modified object code on the User Product (for example, the work has | ||||||
|  | been installed in ROM). | ||||||
|  |  | ||||||
|  |   The requirement to provide Installation Information does not include a | ||||||
|  | requirement to continue to provide support service, warranty, or updates | ||||||
|  | for a work that has been modified or installed by the recipient, or for | ||||||
|  | the User Product in which it has been modified or installed.  Access to a | ||||||
|  | network may be denied when the modification itself materially and | ||||||
|  | adversely affects the operation of the network or violates the rules and | ||||||
|  | protocols for communication across the network. | ||||||
|  |  | ||||||
|  |   Corresponding Source conveyed, and Installation Information provided, | ||||||
|  | in accord with this section must be in a format that is publicly | ||||||
|  | documented (and with an implementation available to the public in | ||||||
|  | source code form), and must require no special password or key for | ||||||
|  | unpacking, reading or copying. | ||||||
|  |  | ||||||
|  |   7. Additional Terms. | ||||||
|  |  | ||||||
|  |   "Additional permissions" are terms that supplement the terms of this | ||||||
|  | License by making exceptions from one or more of its conditions. | ||||||
|  | Additional permissions that are applicable to the entire Program shall | ||||||
|  | be treated as though they were included in this License, to the extent | ||||||
|  | that they are valid under applicable law.  If additional permissions | ||||||
|  | apply only to part of the Program, that part may be used separately | ||||||
|  | under those permissions, but the entire Program remains governed by | ||||||
|  | this License without regard to the additional permissions. | ||||||
|  |  | ||||||
|  |   When you convey a copy of a covered work, you may at your option | ||||||
|  | remove any additional permissions from that copy, or from any part of | ||||||
|  | it.  (Additional permissions may be written to require their own | ||||||
|  | removal in certain cases when you modify the work.)  You may place | ||||||
|  | additional permissions on material, added by you to a covered work, | ||||||
|  | for which you have or can give appropriate copyright permission. | ||||||
|  |  | ||||||
|  |   Notwithstanding any other provision of this License, for material you | ||||||
|  | add to a covered work, you may (if authorized by the copyright holders of | ||||||
|  | that material) supplement the terms of this License with terms: | ||||||
|  |  | ||||||
|  |     a) Disclaiming warranty or limiting liability differently from the | ||||||
|  |     terms of sections 15 and 16 of this License; or | ||||||
|  |  | ||||||
|  |     b) Requiring preservation of specified reasonable legal notices or | ||||||
|  |     author attributions in that material or in the Appropriate Legal | ||||||
|  |     Notices displayed by works containing it; or | ||||||
|  |  | ||||||
|  |     c) Prohibiting misrepresentation of the origin of that material, or | ||||||
|  |     requiring that modified versions of such material be marked in | ||||||
|  |     reasonable ways as different from the original version; or | ||||||
|  |  | ||||||
|  |     d) Limiting the use for publicity purposes of names of licensors or | ||||||
|  |     authors of the material; or | ||||||
|  |  | ||||||
|  |     e) Declining to grant rights under trademark law for use of some | ||||||
|  |     trade names, trademarks, or service marks; or | ||||||
|  |  | ||||||
|  |     f) Requiring indemnification of licensors and authors of that | ||||||
|  |     material by anyone who conveys the material (or modified versions of | ||||||
|  |     it) with contractual assumptions of liability to the recipient, for | ||||||
|  |     any liability that these contractual assumptions directly impose on | ||||||
|  |     those licensors and authors. | ||||||
|  |  | ||||||
|  |   All other non-permissive additional terms are considered "further | ||||||
|  | restrictions" within the meaning of section 10.  If the Program as you | ||||||
|  | received it, or any part of it, contains a notice stating that it is | ||||||
|  | governed by this License along with a term that is a further | ||||||
|  | restriction, you may remove that term.  If a license document contains | ||||||
|  | a further restriction but permits relicensing or conveying under this | ||||||
|  | License, you may add to a covered work material governed by the terms | ||||||
|  | of that license document, provided that the further restriction does | ||||||
|  | not survive such relicensing or conveying. | ||||||
|  |  | ||||||
|  |   If you add terms to a covered work in accord with this section, you | ||||||
|  | must place, in the relevant source files, a statement of the | ||||||
|  | additional terms that apply to those files, or a notice indicating | ||||||
|  | where to find the applicable terms. | ||||||
|  |  | ||||||
|  |   Additional terms, permissive or non-permissive, may be stated in the | ||||||
|  | form of a separately written license, or stated as exceptions; | ||||||
|  | the above requirements apply either way. | ||||||
|  |  | ||||||
|  |   8. Termination. | ||||||
|  |  | ||||||
|  |   You may not propagate or modify a covered work except as expressly | ||||||
|  | provided under this License.  Any attempt otherwise to propagate or | ||||||
|  | modify it is void, and will automatically terminate your rights under | ||||||
|  | this License (including any patent licenses granted under the third | ||||||
|  | paragraph of section 11). | ||||||
|  |  | ||||||
|  |   However, if you cease all violation of this License, then your | ||||||
|  | license from a particular copyright holder is reinstated (a) | ||||||
|  | provisionally, unless and until the copyright holder explicitly and | ||||||
|  | finally terminates your license, and (b) permanently, if the copyright | ||||||
|  | holder fails to notify you of the violation by some reasonable means | ||||||
|  | prior to 60 days after the cessation. | ||||||
|  |  | ||||||
|  |   Moreover, your license from a particular copyright holder is | ||||||
|  | reinstated permanently if the copyright holder notifies you of the | ||||||
|  | violation by some reasonable means, this is the first time you have | ||||||
|  | received notice of violation of this License (for any work) from that | ||||||
|  | copyright holder, and you cure the violation prior to 30 days after | ||||||
|  | your receipt of the notice. | ||||||
|  |  | ||||||
|  |   Termination of your rights under this section does not terminate the | ||||||
|  | licenses of parties who have received copies or rights from you under | ||||||
|  | this License.  If your rights have been terminated and not permanently | ||||||
|  | reinstated, you do not qualify to receive new licenses for the same | ||||||
|  | material under section 10. | ||||||
|  |  | ||||||
|  |   9. Acceptance Not Required for Having Copies. | ||||||
|  |  | ||||||
|  |   You are not required to accept this License in order to receive or | ||||||
|  | run a copy of the Program.  Ancillary propagation of a covered work | ||||||
|  | occurring solely as a consequence of using peer-to-peer transmission | ||||||
|  | to receive a copy likewise does not require acceptance.  However, | ||||||
|  | nothing other than this License grants you permission to propagate or | ||||||
|  | modify any covered work.  These actions infringe copyright if you do | ||||||
|  | not accept this License.  Therefore, by modifying or propagating a | ||||||
|  | covered work, you indicate your acceptance of this License to do so. | ||||||
|  |  | ||||||
|  |   10. Automatic Licensing of Downstream Recipients. | ||||||
|  |  | ||||||
|  |   Each time you convey a covered work, the recipient automatically | ||||||
|  | receives a license from the original licensors, to run, modify and | ||||||
|  | propagate that work, subject to this License.  You are not responsible | ||||||
|  | for enforcing compliance by third parties with this License. | ||||||
|  |  | ||||||
|  |   An "entity transaction" is a transaction transferring control of an | ||||||
|  | organization, or substantially all assets of one, or subdividing an | ||||||
|  | organization, or merging organizations.  If propagation of a covered | ||||||
|  | work results from an entity transaction, each party to that | ||||||
|  | transaction who receives a copy of the work also receives whatever | ||||||
|  | licenses to the work the party's predecessor in interest had or could | ||||||
|  | give under the previous paragraph, plus a right to possession of the | ||||||
|  | Corresponding Source of the work from the predecessor in interest, if | ||||||
|  | the predecessor has it or can get it with reasonable efforts. | ||||||
|  |  | ||||||
|  |   You may not impose any further restrictions on the exercise of the | ||||||
|  | rights granted or affirmed under this License.  For example, you may | ||||||
|  | not impose a license fee, royalty, or other charge for exercise of | ||||||
|  | rights granted under this License, and you may not initiate litigation | ||||||
|  | (including a cross-claim or counterclaim in a lawsuit) alleging that | ||||||
|  | any patent claim is infringed by making, using, selling, offering for | ||||||
|  | sale, or importing the Program or any portion of it. | ||||||
|  |  | ||||||
|  |   11. Patents. | ||||||
|  |  | ||||||
|  |   A "contributor" is a copyright holder who authorizes use under this | ||||||
|  | License of the Program or a work on which the Program is based.  The | ||||||
|  | work thus licensed is called the contributor's "contributor version". | ||||||
|  |  | ||||||
|  |   A contributor's "essential patent claims" are all patent claims | ||||||
|  | owned or controlled by the contributor, whether already acquired or | ||||||
|  | hereafter acquired, that would be infringed by some manner, permitted | ||||||
|  | by this License, of making, using, or selling its contributor version, | ||||||
|  | but do not include claims that would be infringed only as a | ||||||
|  | consequence of further modification of the contributor version.  For | ||||||
|  | purposes of this definition, "control" includes the right to grant | ||||||
|  | patent sublicenses in a manner consistent with the requirements of | ||||||
|  | this License. | ||||||
|  |  | ||||||
|  |   Each contributor grants you a non-exclusive, worldwide, royalty-free | ||||||
|  | patent license under the contributor's essential patent claims, to | ||||||
|  | make, use, sell, offer for sale, import and otherwise run, modify and | ||||||
|  | propagate the contents of its contributor version. | ||||||
|  |  | ||||||
|  |   In the following three paragraphs, a "patent license" is any express | ||||||
|  | agreement or commitment, however denominated, not to enforce a patent | ||||||
|  | (such as an express permission to practice a patent or covenant not to | ||||||
|  | sue for patent infringement).  To "grant" such a patent license to a | ||||||
|  | party means to make such an agreement or commitment not to enforce a | ||||||
|  | patent against the party. | ||||||
|  |  | ||||||
|  |   If you convey a covered work, knowingly relying on a patent license, | ||||||
|  | and the Corresponding Source of the work is not available for anyone | ||||||
|  | to copy, free of charge and under the terms of this License, through a | ||||||
|  | publicly available network server or other readily accessible means, | ||||||
|  | then you must either (1) cause the Corresponding Source to be so | ||||||
|  | available, or (2) arrange to deprive yourself of the benefit of the | ||||||
|  | patent license for this particular work, or (3) arrange, in a manner | ||||||
|  | consistent with the requirements of this License, to extend the patent | ||||||
|  | license to downstream recipients.  "Knowingly relying" means you have | ||||||
|  | actual knowledge that, but for the patent license, your conveying the | ||||||
|  | covered work in a country, or your recipient's use of the covered work | ||||||
|  | in a country, would infringe one or more identifiable patents in that | ||||||
|  | country that you have reason to believe are valid. | ||||||
|  |  | ||||||
|  |   If, pursuant to or in connection with a single transaction or | ||||||
|  | arrangement, you convey, or propagate by procuring conveyance of, a | ||||||
|  | covered work, and grant a patent license to some of the parties | ||||||
|  | receiving the covered work authorizing them to use, propagate, modify | ||||||
|  | or convey a specific copy of the covered work, then the patent license | ||||||
|  | you grant is automatically extended to all recipients of the covered | ||||||
|  | work and works based on it. | ||||||
|  |  | ||||||
|  |   A patent license is "discriminatory" if it does not include within | ||||||
|  | the scope of its coverage, prohibits the exercise of, or is | ||||||
|  | conditioned on the non-exercise of one or more of the rights that are | ||||||
|  | specifically granted under this License.  You may not convey a covered | ||||||
|  | work if you are a party to an arrangement with a third party that is | ||||||
|  | in the business of distributing software, under which you make payment | ||||||
|  | to the third party based on the extent of your activity of conveying | ||||||
|  | the work, and under which the third party grants, to any of the | ||||||
|  | parties who would receive the covered work from you, a discriminatory | ||||||
|  | patent license (a) in connection with copies of the covered work | ||||||
|  | conveyed by you (or copies made from those copies), or (b) primarily | ||||||
|  | for and in connection with specific products or compilations that | ||||||
|  | contain the covered work, unless you entered into that arrangement, | ||||||
|  | or that patent license was granted, prior to 28 March 2007. | ||||||
|  |  | ||||||
|  |   Nothing in this License shall be construed as excluding or limiting | ||||||
|  | any implied license or other defenses to infringement that may | ||||||
|  | otherwise be available to you under applicable patent law. | ||||||
|  |  | ||||||
|  |   12. No Surrender of Others' Freedom. | ||||||
|  |  | ||||||
|  |   If conditions are imposed on you (whether by court order, agreement or | ||||||
|  | otherwise) that contradict the conditions of this License, they do not | ||||||
|  | excuse you from the conditions of this License.  If you cannot convey a | ||||||
|  | covered work so as to satisfy simultaneously your obligations under this | ||||||
|  | License and any other pertinent obligations, then as a consequence you may | ||||||
|  | not convey it at all.  For example, if you agree to terms that obligate you | ||||||
|  | to collect a royalty for further conveying from those to whom you convey | ||||||
|  | the Program, the only way you could satisfy both those terms and this | ||||||
|  | License would be to refrain entirely from conveying the Program. | ||||||
|  |  | ||||||
|  |   13. Use with the GNU Affero General Public License. | ||||||
|  |  | ||||||
|  |   Notwithstanding any other provision of this License, you have | ||||||
|  | permission to link or combine any covered work with a work licensed | ||||||
|  | under version 3 of the GNU Affero General Public License into a single | ||||||
|  | combined work, and to convey the resulting work.  The terms of this | ||||||
|  | License will continue to apply to the part which is the covered work, | ||||||
|  | but the special requirements of the GNU Affero General Public License, | ||||||
|  | section 13, concerning interaction through a network will apply to the | ||||||
|  | combination as such. | ||||||
|  |  | ||||||
|  |   14. Revised Versions of this License. | ||||||
|  |  | ||||||
|  |   The Free Software Foundation may publish revised and/or new versions of | ||||||
|  | the GNU General Public License from time to time.  Such new versions will | ||||||
|  | be similar in spirit to the present version, but may differ in detail to | ||||||
|  | address new problems or concerns. | ||||||
|  |  | ||||||
|  |   Each version is given a distinguishing version number.  If the | ||||||
|  | Program specifies that a certain numbered version of the GNU General | ||||||
|  | Public License "or any later version" applies to it, you have the | ||||||
|  | option of following the terms and conditions either of that numbered | ||||||
|  | version or of any later version published by the Free Software | ||||||
|  | Foundation.  If the Program does not specify a version number of the | ||||||
|  | GNU General Public License, you may choose any version ever published | ||||||
|  | by the Free Software Foundation. | ||||||
|  |  | ||||||
|  |   If the Program specifies that a proxy can decide which future | ||||||
|  | versions of the GNU General Public License can be used, that proxy's | ||||||
|  | public statement of acceptance of a version permanently authorizes you | ||||||
|  | to choose that version for the Program. | ||||||
|  |  | ||||||
|  |   Later license versions may give you additional or different | ||||||
|  | permissions.  However, no additional obligations are imposed on any | ||||||
|  | author or copyright holder as a result of your choosing to follow a | ||||||
|  | later version. | ||||||
|  |  | ||||||
|  |   15. Disclaimer of Warranty. | ||||||
|  |  | ||||||
|  |   THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY | ||||||
|  | APPLICABLE LAW.  EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT | ||||||
|  | HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY | ||||||
|  | OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, | ||||||
|  | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR | ||||||
|  | PURPOSE.  THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM | ||||||
|  | IS WITH YOU.  SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF | ||||||
|  | ALL NECESSARY SERVICING, REPAIR OR CORRECTION. | ||||||
|  |  | ||||||
|  |   16. Limitation of Liability. | ||||||
|  |  | ||||||
|  |   IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING | ||||||
|  | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS | ||||||
|  | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY | ||||||
|  | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE | ||||||
|  | USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF | ||||||
|  | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD | ||||||
|  | PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), | ||||||
|  | EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF | ||||||
|  | SUCH DAMAGES. | ||||||
|  |  | ||||||
|  |   17. Interpretation of Sections 15 and 16. | ||||||
|  |  | ||||||
|  |   If the disclaimer of warranty and limitation of liability provided | ||||||
|  | above cannot be given local legal effect according to their terms, | ||||||
|  | reviewing courts shall apply local law that most closely approximates | ||||||
|  | an absolute waiver of all civil liability in connection with the | ||||||
|  | Program, unless a warranty or assumption of liability accompanies a | ||||||
|  | copy of the Program in return for a fee. | ||||||
|  |  | ||||||
|  |                      END OF TERMS AND CONDITIONS | ||||||
|  |  | ||||||
|  |             How to Apply These Terms to Your New Programs | ||||||
|  |  | ||||||
|  |   If you develop a new program, and you want it to be of the greatest | ||||||
|  | possible use to the public, the best way to achieve this is to make it | ||||||
|  | free software which everyone can redistribute and change under these terms. | ||||||
|  |  | ||||||
|  |   To do so, attach the following notices to the program.  It is safest | ||||||
|  | to attach them to the start of each source file to most effectively | ||||||
|  | state the exclusion of warranty; and each file should have at least | ||||||
|  | the "copyright" line and a pointer to where the full notice is found. | ||||||
|  |  | ||||||
|  |     <one line to give the program's name and a brief idea of what it does.> | ||||||
|  |     Copyright (C) <year>  <name of author> | ||||||
|  |  | ||||||
|  |     This program is free software: you can redistribute it and/or modify | ||||||
|  |     it under the terms of the GNU General Public License as published by | ||||||
|  |     the Free Software Foundation, either version 3 of the License, or | ||||||
|  |     (at your option) any later version. | ||||||
|  |  | ||||||
|  |     This program is distributed in the hope that it will be useful, | ||||||
|  |     but WITHOUT ANY WARRANTY; without even the implied warranty of | ||||||
|  |     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the | ||||||
|  |     GNU General Public License for more details. | ||||||
|  |  | ||||||
|  |     You should have received a copy of the GNU General Public License | ||||||
|  |     along with this program.  If not, see <https://www.gnu.org/licenses/>. | ||||||
|  |  | ||||||
|  | Also add information on how to contact you by electronic and paper mail. | ||||||
|  |  | ||||||
|  |   If the program does terminal interaction, make it output a short | ||||||
|  | notice like this when it starts in an interactive mode: | ||||||
|  |  | ||||||
|  |     <program>  Copyright (C) <year>  <name of author> | ||||||
|  |     This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. | ||||||
|  |     This is free software, and you are welcome to redistribute it | ||||||
|  |     under certain conditions; type `show c' for details. | ||||||
|  |  | ||||||
|  | The hypothetical commands `show w' and `show c' should show the appropriate | ||||||
|  | parts of the General Public License.  Of course, your program's commands | ||||||
|  | might be different; for a GUI interface, you would use an "about box". | ||||||
|  |  | ||||||
|  |   You should also get your employer (if you work as a programmer) or school, | ||||||
|  | if any, to sign a "copyright disclaimer" for the program, if necessary. | ||||||
|  | For more information on this, and how to apply and follow the GNU GPL, see | ||||||
|  | <https://www.gnu.org/licenses/>. | ||||||
|  |  | ||||||
|  |   The GNU General Public License does not permit incorporating your program | ||||||
|  | into proprietary programs.  If your program is a subroutine library, you | ||||||
|  | may consider it more useful to permit linking proprietary applications with | ||||||
|  | the library.  If this is what you want to do, use the GNU Lesser General | ||||||
|  | Public License instead of this License.  But first, please read | ||||||
|  | <https://www.gnu.org/licenses/why-not-lgpl.html>. | ||||||
							
								
								
									
										165
									
								
								COPYING.LESSER
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										165
									
								
								COPYING.LESSER
									
									
									
									
									
										Normal file
									
								
							| @@ -0,0 +1,165 @@ | |||||||
|  |                    GNU LESSER GENERAL PUBLIC LICENSE | ||||||
|  |                        Version 3, 29 June 2007 | ||||||
|  |  | ||||||
|  |  Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/> | ||||||
|  |  Everyone is permitted to copy and distribute verbatim copies | ||||||
|  |  of this license document, but changing it is not allowed. | ||||||
|  |  | ||||||
|  |  | ||||||
|  |   This version of the GNU Lesser General Public License incorporates | ||||||
|  | the terms and conditions of version 3 of the GNU General Public | ||||||
|  | License, supplemented by the additional permissions listed below. | ||||||
|  |  | ||||||
|  |   0. Additional Definitions. | ||||||
|  |  | ||||||
|  |   As used herein, "this License" refers to version 3 of the GNU Lesser | ||||||
|  | General Public License, and the "GNU GPL" refers to version 3 of the GNU | ||||||
|  | General Public License. | ||||||
|  |  | ||||||
|  |   "The Library" refers to a covered work governed by this License, | ||||||
|  | other than an Application or a Combined Work as defined below. | ||||||
|  |  | ||||||
|  |   An "Application" is any work that makes use of an interface provided | ||||||
|  | by the Library, but which is not otherwise based on the Library. | ||||||
|  | Defining a subclass of a class defined by the Library is deemed a mode | ||||||
|  | of using an interface provided by the Library. | ||||||
|  |  | ||||||
|  |   A "Combined Work" is a work produced by combining or linking an | ||||||
|  | Application with the Library.  The particular version of the Library | ||||||
|  | with which the Combined Work was made is also called the "Linked | ||||||
|  | Version". | ||||||
|  |  | ||||||
|  |   The "Minimal Corresponding Source" for a Combined Work means the | ||||||
|  | Corresponding Source for the Combined Work, excluding any source code | ||||||
|  | for portions of the Combined Work that, considered in isolation, are | ||||||
|  | based on the Application, and not on the Linked Version. | ||||||
|  |  | ||||||
|  |   The "Corresponding Application Code" for a Combined Work means the | ||||||
|  | object code and/or source code for the Application, including any data | ||||||
|  | and utility programs needed for reproducing the Combined Work from the | ||||||
|  | Application, but excluding the System Libraries of the Combined Work. | ||||||
|  |  | ||||||
|  |   1. Exception to Section 3 of the GNU GPL. | ||||||
|  |  | ||||||
|  |   You may convey a covered work under sections 3 and 4 of this License | ||||||
|  | without being bound by section 3 of the GNU GPL. | ||||||
|  |  | ||||||
|  |   2. Conveying Modified Versions. | ||||||
|  |  | ||||||
|  |   If you modify a copy of the Library, and, in your modifications, a | ||||||
|  | facility refers to a function or data to be supplied by an Application | ||||||
|  | that uses the facility (other than as an argument passed when the | ||||||
|  | facility is invoked), then you may convey a copy of the modified | ||||||
|  | version: | ||||||
|  |  | ||||||
|  |    a) under this License, provided that you make a good faith effort to | ||||||
|  |    ensure that, in the event an Application does not supply the | ||||||
|  |    function or data, the facility still operates, and performs | ||||||
|  |    whatever part of its purpose remains meaningful, or | ||||||
|  |  | ||||||
|  |    b) under the GNU GPL, with none of the additional permissions of | ||||||
|  |    this License applicable to that copy. | ||||||
|  |  | ||||||
|  |   3. Object Code Incorporating Material from Library Header Files. | ||||||
|  |  | ||||||
|  |   The object code form of an Application may incorporate material from | ||||||
|  | a header file that is part of the Library.  You may convey such object | ||||||
|  | code under terms of your choice, provided that, if the incorporated | ||||||
|  | material is not limited to numerical parameters, data structure | ||||||
|  | layouts and accessors, or small macros, inline functions and templates | ||||||
|  | (ten or fewer lines in length), you do both of the following: | ||||||
|  |  | ||||||
|  |    a) Give prominent notice with each copy of the object code that the | ||||||
|  |    Library is used in it and that the Library and its use are | ||||||
|  |    covered by this License. | ||||||
|  |  | ||||||
|  |    b) Accompany the object code with a copy of the GNU GPL and this license | ||||||
|  |    document. | ||||||
|  |  | ||||||
|  |   4. Combined Works. | ||||||
|  |  | ||||||
|  |   You may convey a Combined Work under terms of your choice that, | ||||||
|  | taken together, effectively do not restrict modification of the | ||||||
|  | portions of the Library contained in the Combined Work and reverse | ||||||
|  | engineering for debugging such modifications, if you also do each of | ||||||
|  | the following: | ||||||
|  |  | ||||||
|  |    a) Give prominent notice with each copy of the Combined Work that | ||||||
|  |    the Library is used in it and that the Library and its use are | ||||||
|  |    covered by this License. | ||||||
|  |  | ||||||
|  |    b) Accompany the Combined Work with a copy of the GNU GPL and this license | ||||||
|  |    document. | ||||||
|  |  | ||||||
|  |    c) For a Combined Work that displays copyright notices during | ||||||
|  |    execution, include the copyright notice for the Library among | ||||||
|  |    these notices, as well as a reference directing the user to the | ||||||
|  |    copies of the GNU GPL and this license document. | ||||||
|  |  | ||||||
|  |    d) Do one of the following: | ||||||
|  |  | ||||||
|  |        0) Convey the Minimal Corresponding Source under the terms of this | ||||||
|  |        License, and the Corresponding Application Code in a form | ||||||
|  |        suitable for, and under terms that permit, the user to | ||||||
|  |        recombine or relink the Application with a modified version of | ||||||
|  |        the Linked Version to produce a modified Combined Work, in the | ||||||
|  |        manner specified by section 6 of the GNU GPL for conveying | ||||||
|  |        Corresponding Source. | ||||||
|  |  | ||||||
|  |        1) Use a suitable shared library mechanism for linking with the | ||||||
|  |        Library.  A suitable mechanism is one that (a) uses at run time | ||||||
|  |        a copy of the Library already present on the user's computer | ||||||
|  |        system, and (b) will operate properly with a modified version | ||||||
|  |        of the Library that is interface-compatible with the Linked | ||||||
|  |        Version. | ||||||
|  |  | ||||||
|  |    e) Provide Installation Information, but only if you would otherwise | ||||||
|  |    be required to provide such information under section 6 of the | ||||||
|  |    GNU GPL, and only to the extent that such information is | ||||||
|  |    necessary to install and execute a modified version of the | ||||||
|  |    Combined Work produced by recombining or relinking the | ||||||
|  |    Application with a modified version of the Linked Version. (If | ||||||
|  |    you use option 4d0, the Installation Information must accompany | ||||||
|  |    the Minimal Corresponding Source and Corresponding Application | ||||||
|  |    Code. If you use option 4d1, you must provide the Installation | ||||||
|  |    Information in the manner specified by section 6 of the GNU GPL | ||||||
|  |    for conveying Corresponding Source.) | ||||||
|  |  | ||||||
|  |   5. Combined Libraries. | ||||||
|  |  | ||||||
|  |   You may place library facilities that are a work based on the | ||||||
|  | Library side by side in a single library together with other library | ||||||
|  | facilities that are not Applications and are not covered by this | ||||||
|  | License, and convey such a combined library under terms of your | ||||||
|  | choice, if you do both of the following: | ||||||
|  |  | ||||||
|  |    a) Accompany the combined library with a copy of the same work based | ||||||
|  |    on the Library, uncombined with any other library facilities, | ||||||
|  |    conveyed under the terms of this License. | ||||||
|  |  | ||||||
|  |    b) Give prominent notice with the combined library that part of it | ||||||
|  |    is a work based on the Library, and explaining where to find the | ||||||
|  |    accompanying uncombined form of the same work. | ||||||
|  |  | ||||||
|  |   6. Revised Versions of the GNU Lesser General Public License. | ||||||
|  |  | ||||||
|  |   The Free Software Foundation may publish revised and/or new versions | ||||||
|  | of the GNU Lesser General Public License from time to time. Such new | ||||||
|  | versions will be similar in spirit to the present version, but may | ||||||
|  | differ in detail to address new problems or concerns. | ||||||
|  |  | ||||||
|  |   Each version is given a distinguishing version number. If the | ||||||
|  | Library as you received it specifies that a certain numbered version | ||||||
|  | of the GNU Lesser General Public License "or any later version" | ||||||
|  | applies to it, you have the option of following the terms and | ||||||
|  | conditions either of that published version or of any later version | ||||||
|  | published by the Free Software Foundation. If the Library as you | ||||||
|  | received it does not specify a version number of the GNU Lesser | ||||||
|  | General Public License, you may choose any version of the GNU Lesser | ||||||
|  | General Public License ever published by the Free Software Foundation. | ||||||
|  |  | ||||||
|  |   If the Library as you received it specifies that a proxy can decide | ||||||
|  | whether future versions of the GNU Lesser General Public License shall | ||||||
|  | apply, that proxy's public statement of acceptance of any version is | ||||||
|  | permanent authorization for you to choose that version for the | ||||||
|  | Library. | ||||||
							
								
								
									
										52
									
								
								README.md
									
									
									
									
									
								
							
							
						
						
									
										52
									
								
								README.md
									
									
									
									
									
								
							| @@ -1,19 +1,61 @@ | |||||||
| # asyncio-taskpool | # asyncio-taskpool | ||||||
|  |  | ||||||
| Dynamically manage pools of asyncio tasks | **Dynamically manage pools of asyncio tasks** | ||||||
|  |  | ||||||
|  | ## Summary | ||||||
|  |  | ||||||
|  | A task pool is an object with a simple interface for aggregating and dynamically managing asynchronous tasks. | ||||||
|  |  | ||||||
|  | With an interface that is intentionally similar to the [`multiprocessing.Pool`](https://docs.python.org/3/library/multiprocessing.html#module-multiprocessing.pool) class from the standard library, the `TaskPool` provides you such methods as `apply`, `map`, and `starmap` to execute coroutines concurrently as [`asyncio.Task`](https://docs.python.org/3/library/asyncio-task.html#task-object) objects. There is no limitation imposed on what kind of tasks can be run or in what combination, when new ones can be added, or when they can be cancelled. | ||||||
|  |  | ||||||
|  | For a more streamlined use-case, the `SimpleTaskPool` provides an even more intuitive and simple interface at the cost of flexibility. | ||||||
|  |  | ||||||
|  | If you need control over a task pool at runtime, you can launch an asynchronous `ControlServer` to be able to interface with the pool from an outside process or via a network, and stop/start tasks within the pool as you wish. | ||||||
|  |  | ||||||
| ## Usage | ## Usage | ||||||
|  |  | ||||||
| See [USAGE.md](usage/USAGE.md) | Generally speaking, a task is added to a pool by providing it with a coroutine function reference as well as the arguments for that function. Here is what that could look like: | ||||||
|  | ```python | ||||||
|  | from asyncio_taskpool import SimpleTaskPool | ||||||
|  | ... | ||||||
|  | async def work(foo, bar): ... | ||||||
|  | ... | ||||||
|  | async def main(): | ||||||
|  |     pool = SimpleTaskPool(work, args=('xyz', 420)) | ||||||
|  |     await pool.start(5) | ||||||
|  |     ... | ||||||
|  |     pool.stop(3) | ||||||
|  |     ... | ||||||
|  |     pool.lock() | ||||||
|  |     await pool.gather() | ||||||
|  |     ... | ||||||
|  | ``` | ||||||
|  |  | ||||||
|  | Since one of the main goals of `asyncio-taskpool` is to be able to start/stop tasks dynamically or "on-the-fly", _most_ of the associated methods are non-blocking _most_ of the time. A notable exception is the `gather` method for awaiting the return of all tasks in the pool. (It is essentially a glorified wrapper around the [`asyncio.gather`](https://docs.python.org/3/library/asyncio-task.html#asyncio.gather) function.) | ||||||
|  |  | ||||||
|  | For working and fully documented demo scripts see [USAGE.md](usage/USAGE.md). | ||||||
|  |  | ||||||
| ## Installation | ## Installation | ||||||
|  |  | ||||||
| `pip install asyncio-taskpool` | ```shell | ||||||
|  | pip install asyncio-taskpool | ||||||
|  | ``` | ||||||
|  |  | ||||||
| ## Dependencies | ## Dependencies | ||||||
|  |  | ||||||
| Python Version 3.8+, tested on Linux | Python Version 3.8+, tested on Linux | ||||||
|  |  | ||||||
| ## Building from source | ## Testing | ||||||
|  |  | ||||||
| Run `python -m build` | Install `asyncio-taskpool[dev]` dependencies or just manually install `coverage` with `pip`.  | ||||||
|  | Execute the [`./coverage.sh`](coverage.sh) shell script to run all unit tests and receive the coverage report. | ||||||
|  |  | ||||||
|  | ## License | ||||||
|  |  | ||||||
|  | `asyncio-taskpool` is licensed under the **GNU LGPL version 3.0** specifically. | ||||||
|  |  | ||||||
|  | The full license texts for the [GNU GPLv3.0](COPYING) and the [GNU LGPLv3.0](COPYING.LESSER) are included in this repository. If not, see https://www.gnu.org/licenses/. | ||||||
|  |  | ||||||
|  | ## Copyright | ||||||
|  |  | ||||||
|  | © 2022 Daniil Fajnberg | ||||||
|   | |||||||
							
								
								
									
										10
									
								
								setup.cfg
									
									
									
									
									
								
							
							
						
						
									
										10
									
								
								setup.cfg
									
									
									
									
									
								
							| @@ -1,17 +1,25 @@ | |||||||
| [metadata] | [metadata] | ||||||
| name = asyncio-taskpool | name = asyncio-taskpool | ||||||
| version = 0.1.7 | version = 0.3.2 | ||||||
| author = Daniil Fajnberg | author = Daniil Fajnberg | ||||||
| author_email = mail@daniil.fajnberg.de | author_email = mail@daniil.fajnberg.de | ||||||
| description = Dynamically manage pools of asyncio tasks | description = Dynamically manage pools of asyncio tasks | ||||||
| long_description = file: README.md | long_description = file: README.md | ||||||
| long_description_content_type = text/markdown | long_description_content_type = text/markdown | ||||||
|  | keywords = asyncio, concurrency, tasks, coroutines, asynchronous, server | ||||||
| url = https://git.fajnberg.de/daniil/asyncio-taskpool | url = https://git.fajnberg.de/daniil/asyncio-taskpool | ||||||
| project_urls = | project_urls = | ||||||
|     Bug Tracker = https://git.fajnberg.de/daniil/asyncio-taskpool/issues |     Bug Tracker = https://git.fajnberg.de/daniil/asyncio-taskpool/issues | ||||||
| classifiers = | classifiers = | ||||||
|  |     Development Status :: 3 - Alpha | ||||||
|     Programming Language :: Python :: 3 |     Programming Language :: Python :: 3 | ||||||
|     Operating System :: OS Independent |     Operating System :: OS Independent | ||||||
|  |     License :: OSI Approved :: GNU Lesser General Public License v3 (LGPLv3) | ||||||
|  |     Intended Audience :: Developers | ||||||
|  |     Intended Audience :: System Administrators | ||||||
|  |     Framework :: AsyncIO | ||||||
|  |     Topic :: Software Development :: Libraries | ||||||
|  |     Topic :: Software Development :: Libraries :: Python Modules | ||||||
|  |  | ||||||
| [options] | [options] | ||||||
| package_dir = | package_dir = | ||||||
|   | |||||||
| @@ -1,2 +1,23 @@ | |||||||
|  | __author__ = "Daniil Fajnberg" | ||||||
|  | __copyright__ = "Copyright © 2022 Daniil Fajnberg" | ||||||
|  | __license__ = """GNU LGPLv3.0 | ||||||
|  |  | ||||||
|  | This file is part of asyncio-taskpool. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is free software: you can redistribute it and/or modify it under the terms of | ||||||
|  | version 3.0 of the GNU Lesser General Public License as published by the Free Software Foundation. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;  | ||||||
|  | without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  | ||||||
|  | See the GNU Lesser General Public License for more details. | ||||||
|  |  | ||||||
|  | You should have received a copy of the GNU Lesser General Public License along with asyncio-taskpool.  | ||||||
|  | If not, see <https://www.gnu.org/licenses/>.""" | ||||||
|  |  | ||||||
|  | __doc__ = """ | ||||||
|  | Brings the main classes up to package level for import convenience. | ||||||
|  | """ | ||||||
|  |  | ||||||
|  |  | ||||||
| from .pool import TaskPool, SimpleTaskPool | from .pool import TaskPool, SimpleTaskPool | ||||||
| from .server import UnixControlServer | from .server import UnixControlServer | ||||||
|   | |||||||
| @@ -1,3 +1,24 @@ | |||||||
|  | __author__ = "Daniil Fajnberg" | ||||||
|  | __copyright__ = "Copyright © 2022 Daniil Fajnberg" | ||||||
|  | __license__ = """GNU LGPLv3.0 | ||||||
|  |  | ||||||
|  | This file is part of asyncio-taskpool. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is free software: you can redistribute it and/or modify it under the terms of | ||||||
|  | version 3.0 of the GNU Lesser General Public License as published by the Free Software Foundation. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;  | ||||||
|  | without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  | ||||||
|  | See the GNU Lesser General Public License for more details. | ||||||
|  |  | ||||||
|  | You should have received a copy of the GNU Lesser General Public License along with asyncio-taskpool.  | ||||||
|  | If not, see <https://www.gnu.org/licenses/>.""" | ||||||
|  |  | ||||||
|  | __doc__ = """ | ||||||
|  | CLI client entry point. | ||||||
|  | """ | ||||||
|  |  | ||||||
|  |  | ||||||
| import sys | import sys | ||||||
| from argparse import ArgumentParser | from argparse import ArgumentParser | ||||||
| from asyncio import run | from asyncio import run | ||||||
|   | |||||||
| @@ -1,10 +1,33 @@ | |||||||
|  | __author__ = "Daniil Fajnberg" | ||||||
|  | __copyright__ = "Copyright © 2022 Daniil Fajnberg" | ||||||
|  | __license__ = """GNU LGPLv3.0 | ||||||
|  |  | ||||||
|  | This file is part of asyncio-taskpool. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is free software: you can redistribute it and/or modify it under the terms of | ||||||
|  | version 3.0 of the GNU Lesser General Public License as published by the Free Software Foundation. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;  | ||||||
|  | without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  | ||||||
|  | See the GNU Lesser General Public License for more details. | ||||||
|  |  | ||||||
|  | You should have received a copy of the GNU Lesser General Public License along with asyncio-taskpool.  | ||||||
|  | If not, see <https://www.gnu.org/licenses/>.""" | ||||||
|  |  | ||||||
|  | __doc__ = """ | ||||||
|  | Classes of control clients for a simply interface to a task pool control server. | ||||||
|  | """ | ||||||
|  |  | ||||||
|  |  | ||||||
|  | import json | ||||||
|  | import shutil | ||||||
| import sys | import sys | ||||||
| from abc import ABC, abstractmethod | from abc import ABC, abstractmethod | ||||||
| from asyncio.streams import StreamReader, StreamWriter, open_unix_connection | from asyncio.streams import StreamReader, StreamWriter, open_unix_connection | ||||||
| from pathlib import Path | from pathlib import Path | ||||||
|  |  | ||||||
| from asyncio_taskpool import constants | from .constants import CLIENT_EXIT, CLIENT_INFO, SESSION_MSG_BYTES | ||||||
| from asyncio_taskpool.types import ClientConnT | from .types import ClientConnT | ||||||
|  |  | ||||||
|  |  | ||||||
| class ControlClient(ABC): | class ControlClient(ABC): | ||||||
| @@ -13,19 +36,29 @@ class ControlClient(ABC): | |||||||
|     async def open_connection(self, **kwargs) -> ClientConnT: |     async def open_connection(self, **kwargs) -> ClientConnT: | ||||||
|         raise NotImplementedError |         raise NotImplementedError | ||||||
|  |  | ||||||
|  |     @staticmethod | ||||||
|  |     def client_info() -> dict: | ||||||
|  |         return {CLIENT_INFO.TERMINAL_WIDTH: shutil.get_terminal_size().columns} | ||||||
|  |  | ||||||
|     def __init__(self, **conn_kwargs) -> None: |     def __init__(self, **conn_kwargs) -> None: | ||||||
|         self._conn_kwargs = conn_kwargs |         self._conn_kwargs = conn_kwargs | ||||||
|         self._connected: bool = False |         self._connected: bool = False | ||||||
|  |  | ||||||
|  |     async def _server_handshake(self, reader: StreamReader, writer: StreamWriter) -> None: | ||||||
|  |         self._connected = True | ||||||
|  |         writer.write(json.dumps(self.client_info()).encode()) | ||||||
|  |         await writer.drain() | ||||||
|  |         print("Connected to", (await reader.read(SESSION_MSG_BYTES)).decode()) | ||||||
|  |  | ||||||
|     async def _interact(self, reader: StreamReader, writer: StreamWriter) -> None: |     async def _interact(self, reader: StreamReader, writer: StreamWriter) -> None: | ||||||
|         try: |         try: | ||||||
|             msg = input("> ").strip().lower() |             msg = input("> ").strip().lower() | ||||||
|         except EOFError: |         except EOFError: | ||||||
|             msg = constants.CLIENT_EXIT |             msg = CLIENT_EXIT | ||||||
|         except KeyboardInterrupt: |         except KeyboardInterrupt: | ||||||
|             print() |             print() | ||||||
|             return |             return | ||||||
|         if msg == constants.CLIENT_EXIT: |         if msg == CLIENT_EXIT: | ||||||
|             writer.close() |             writer.close() | ||||||
|             self._connected = False |             self._connected = False | ||||||
|             return |             return | ||||||
| @@ -36,15 +69,14 @@ class ControlClient(ABC): | |||||||
|             self._connected = False |             self._connected = False | ||||||
|             print(e, file=sys.stderr) |             print(e, file=sys.stderr) | ||||||
|             return |             return | ||||||
|         print((await reader.read(constants.MSG_BYTES)).decode()) |         print((await reader.read(SESSION_MSG_BYTES)).decode()) | ||||||
|  |  | ||||||
|     async def start(self): |     async def start(self): | ||||||
|         reader, writer = await self.open_connection(**self._conn_kwargs) |         reader, writer = await self.open_connection(**self._conn_kwargs) | ||||||
|         if reader is None: |         if reader is None: | ||||||
|             print("Failed to connect.", file=sys.stderr) |             print("Failed to connect.", file=sys.stderr) | ||||||
|             return |             return | ||||||
|         self._connected = True |         await self._server_handshake(reader, writer) | ||||||
|         print("Connected to", (await reader.read(constants.MSG_BYTES)).decode()) |  | ||||||
|         while self._connected: |         while self._connected: | ||||||
|             await self._interact(reader, writer) |             await self._interact(reader, writer) | ||||||
|         print("Disconnected from control server.") |         print("Disconnected from control server.") | ||||||
|   | |||||||
| @@ -1,8 +1,44 @@ | |||||||
|  | __author__ = "Daniil Fajnberg" | ||||||
|  | __copyright__ = "Copyright © 2022 Daniil Fajnberg" | ||||||
|  | __license__ = """GNU LGPLv3.0 | ||||||
|  |  | ||||||
|  | This file is part of asyncio-taskpool. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is free software: you can redistribute it and/or modify it under the terms of | ||||||
|  | version 3.0 of the GNU Lesser General Public License as published by the Free Software Foundation. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;  | ||||||
|  | without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  | ||||||
|  | See the GNU Lesser General Public License for more details. | ||||||
|  |  | ||||||
|  | You should have received a copy of the GNU Lesser General Public License along with asyncio-taskpool.  | ||||||
|  | If not, see <https://www.gnu.org/licenses/>.""" | ||||||
|  |  | ||||||
|  | __doc__ = """ | ||||||
|  | Constants used by more than one module in the package. | ||||||
|  | """ | ||||||
|  |  | ||||||
|  |  | ||||||
| PACKAGE_NAME = 'asyncio_taskpool' | PACKAGE_NAME = 'asyncio_taskpool' | ||||||
| MSG_BYTES = 1024 |  | ||||||
| CMD_START = 'start' |  | ||||||
| CMD_STOP = 'stop' |  | ||||||
| CMD_STOP_ALL = 'stop_all' |  | ||||||
| CMD_NUM_RUNNING = 'num_running' |  | ||||||
| CMD_FUNC = 'func' |  | ||||||
| CLIENT_EXIT = 'exit' | CLIENT_EXIT = 'exit' | ||||||
|  |  | ||||||
|  | SESSION_MSG_BYTES = 1024 * 100 | ||||||
|  | SESSION_PARSER_WRITER = 'session_writer' | ||||||
|  |  | ||||||
|  |  | ||||||
|  | class CLIENT_INFO: | ||||||
|  |     __slots__ = () | ||||||
|  |     TERMINAL_WIDTH = 'terminal_width' | ||||||
|  |  | ||||||
|  |  | ||||||
|  | class CMD: | ||||||
|  |     __slots__ = () | ||||||
|  |     CMD = 'command' | ||||||
|  |     NAME = 'name' | ||||||
|  |     POOL_SIZE = 'pool-size' | ||||||
|  |     NUM_RUNNING = 'num-running' | ||||||
|  |     START = 'start' | ||||||
|  |     STOP = 'stop' | ||||||
|  |     STOP_ALL = 'stop-all' | ||||||
|  |     FUNC_NAME = 'func-name' | ||||||
|   | |||||||
| @@ -1,8 +1,29 @@ | |||||||
|  | __author__ = "Daniil Fajnberg" | ||||||
|  | __copyright__ = "Copyright © 2022 Daniil Fajnberg" | ||||||
|  | __license__ = """GNU LGPLv3.0 | ||||||
|  |  | ||||||
|  | This file is part of asyncio-taskpool. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is free software: you can redistribute it and/or modify it under the terms of | ||||||
|  | version 3.0 of the GNU Lesser General Public License as published by the Free Software Foundation. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;  | ||||||
|  | without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  | ||||||
|  | See the GNU Lesser General Public License for more details. | ||||||
|  |  | ||||||
|  | You should have received a copy of the GNU Lesser General Public License along with asyncio-taskpool.  | ||||||
|  | If not, see <https://www.gnu.org/licenses/>.""" | ||||||
|  |  | ||||||
|  | __doc__ = """ | ||||||
|  | Custom exception classes used in various modules. | ||||||
|  | """ | ||||||
|  |  | ||||||
|  |  | ||||||
| class PoolException(Exception): | class PoolException(Exception): | ||||||
|     pass |     pass | ||||||
|  |  | ||||||
|  |  | ||||||
| class PoolIsClosed(PoolException): | class PoolIsLocked(PoolException): | ||||||
|     pass |     pass | ||||||
|  |  | ||||||
|  |  | ||||||
| @@ -22,9 +43,17 @@ class InvalidTaskID(PoolException): | |||||||
|     pass |     pass | ||||||
|  |  | ||||||
|  |  | ||||||
| class PoolStillOpen(PoolException): | class PoolStillUnlocked(PoolException): | ||||||
|     pass |     pass | ||||||
|  |  | ||||||
|  |  | ||||||
| class NotCoroutine(PoolException): | class NotCoroutine(PoolException): | ||||||
|     pass |     pass | ||||||
|  |  | ||||||
|  |  | ||||||
|  | class ServerException(Exception): | ||||||
|  |     pass | ||||||
|  |  | ||||||
|  |  | ||||||
|  | class HelpRequested(ServerException): | ||||||
|  |     pass | ||||||
|   | |||||||
| @@ -1,6 +1,29 @@ | |||||||
|  | __author__ = "Daniil Fajnberg" | ||||||
|  | __copyright__ = "Copyright © 2022 Daniil Fajnberg" | ||||||
|  | __license__ = """GNU LGPLv3.0 | ||||||
|  |  | ||||||
|  | This file is part of asyncio-taskpool. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is free software: you can redistribute it and/or modify it under the terms of | ||||||
|  | version 3.0 of the GNU Lesser General Public License as published by the Free Software Foundation. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;  | ||||||
|  | without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  | ||||||
|  | See the GNU Lesser General Public License for more details. | ||||||
|  |  | ||||||
|  | You should have received a copy of the GNU Lesser General Public License along with asyncio-taskpool.  | ||||||
|  | If not, see <https://www.gnu.org/licenses/>.""" | ||||||
|  |  | ||||||
|  | __doc__ = """ | ||||||
|  | Miscellaneous helper functions. | ||||||
|  | """ | ||||||
|  |  | ||||||
|  |  | ||||||
|  | import re | ||||||
| from asyncio.coroutines import iscoroutinefunction | from asyncio.coroutines import iscoroutinefunction | ||||||
| from asyncio.queues import Queue | from asyncio.queues import Queue | ||||||
| from typing import Any, Optional | from inspect import getdoc | ||||||
|  | from typing import Any, Optional, Union | ||||||
|  |  | ||||||
| from .types import T, AnyCallableT, ArgsT, KwArgsT | from .types import T, AnyCallableT, ArgsT, KwArgsT | ||||||
|  |  | ||||||
| @@ -27,3 +50,21 @@ def star_function(function: AnyCallableT, arg: Any, arg_stars: int = 0) -> T: | |||||||
|  |  | ||||||
| async def join_queue(q: Queue) -> None: | async def join_queue(q: Queue) -> None: | ||||||
|     await q.join() |     await q.join() | ||||||
|  |  | ||||||
|  |  | ||||||
|  | def tasks_str(num: int) -> str: | ||||||
|  |     return "tasks" if num != 1 else "task" | ||||||
|  |  | ||||||
|  |  | ||||||
|  | def get_first_doc_line(obj: object) -> str: | ||||||
|  |     return getdoc(obj).strip().split("\n", 1)[0] | ||||||
|  |  | ||||||
|  |  | ||||||
|  | async def return_or_exception(_function_to_execute: AnyCallableT, *args, **kwargs) -> Union[T, Exception]: | ||||||
|  |     try: | ||||||
|  |         if iscoroutinefunction(_function_to_execute): | ||||||
|  |             return await _function_to_execute(*args, **kwargs) | ||||||
|  |         else: | ||||||
|  |             return _function_to_execute(*args, **kwargs) | ||||||
|  |     except Exception as e: | ||||||
|  |         return e | ||||||
|   | |||||||
| @@ -1,3 +1,36 @@ | |||||||
|  | __author__ = "Daniil Fajnberg" | ||||||
|  | __copyright__ = "Copyright © 2022 Daniil Fajnberg" | ||||||
|  | __license__ = """GNU LGPLv3.0 | ||||||
|  |  | ||||||
|  | This file is part of asyncio-taskpool. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is free software: you can redistribute it and/or modify it under the terms of | ||||||
|  | version 3.0 of the GNU Lesser General Public License as published by the Free Software Foundation. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;  | ||||||
|  | without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  | ||||||
|  | See the GNU Lesser General Public License for more details. | ||||||
|  |  | ||||||
|  | You should have received a copy of the GNU Lesser General Public License along with asyncio-taskpool.  | ||||||
|  | If not, see <https://www.gnu.org/licenses/>.""" | ||||||
|  |  | ||||||
|  | __doc__ = """ | ||||||
|  | This module contains the definitions of the task pool classes. | ||||||
|  |  | ||||||
|  | A task pool is an object with a simple interface for aggregating and dynamically managing asynchronous tasks. | ||||||
|  | Generally speaking, a task is added to a pool by providing it with a coroutine function reference as well as the  | ||||||
|  | arguments for that function. | ||||||
|  |  | ||||||
|  | The `BaseTaskPool` class is a parent class and not intended for direct use. | ||||||
|  | The `TaskPool` and `SimpleTaskPool` are subclasses intended for direct use. | ||||||
|  | While the former allows for heterogeneous collections of tasks that can be entirely unrelated to one another, the  | ||||||
|  | latter requires a preemptive decision about the function **and** its arguments upon initialization and only allows | ||||||
|  | to dynamically control the **number** of tasks running at any point in time. | ||||||
|  |  | ||||||
|  | For further details about the classes check their respective docstrings. | ||||||
|  | """ | ||||||
|  |  | ||||||
|  |  | ||||||
| import logging | import logging | ||||||
| from asyncio import gather | from asyncio import gather | ||||||
| from asyncio.coroutines import iscoroutine, iscoroutinefunction | from asyncio.coroutines import iscoroutine, iscoroutinefunction | ||||||
| @@ -31,7 +64,7 @@ class BaseTaskPool: | |||||||
|         """Initializes the necessary internal attributes and adds the new pool to the general pools list.""" |         """Initializes the necessary internal attributes and adds the new pool to the general pools list.""" | ||||||
|         self._enough_room: Semaphore = Semaphore() |         self._enough_room: Semaphore = Semaphore() | ||||||
|         self.pool_size = pool_size |         self.pool_size = pool_size | ||||||
|         self._open: bool = True |         self._locked: bool = False | ||||||
|         self._counter: int = 0 |         self._counter: int = 0 | ||||||
|         self._running: Dict[int, Task] = {} |         self._running: Dict[int, Task] = {} | ||||||
|         self._cancelled: Dict[int, Task] = {} |         self._cancelled: Dict[int, Task] = {} | ||||||
| @@ -45,6 +78,7 @@ class BaseTaskPool: | |||||||
|         log.debug("%s initialized", str(self)) |         log.debug("%s initialized", str(self)) | ||||||
|  |  | ||||||
|     def __str__(self) -> str: |     def __str__(self) -> str: | ||||||
|  |         """Returns the name of the task pool.""" | ||||||
|         return f'{self.__class__.__name__}-{self._name or self._idx}' |         return f'{self.__class__.__name__}-{self._name or self._idx}' | ||||||
|  |  | ||||||
|     @property |     @property | ||||||
| @@ -71,9 +105,21 @@ class BaseTaskPool: | |||||||
|         self._pool_size = value |         self._pool_size = value | ||||||
|  |  | ||||||
|     @property |     @property | ||||||
|     def is_open(self) -> bool: |     def is_locked(self) -> bool: | ||||||
|         """Returns `True` if more the pool has not been closed yet.""" |         """Returns `True` if more the pool has been locked (see below).""" | ||||||
|         return self._open |         return self._locked | ||||||
|  |  | ||||||
|  |     def lock(self) -> None: | ||||||
|  |         """Disallows any more tasks to be started in the pool.""" | ||||||
|  |         if not self._locked: | ||||||
|  |             self._locked = True | ||||||
|  |             log.info("%s is locked!", str(self)) | ||||||
|  |  | ||||||
|  |     def unlock(self) -> None: | ||||||
|  |         """Allows new tasks to be started in the pool.""" | ||||||
|  |         if self._locked: | ||||||
|  |             self._locked = False | ||||||
|  |             log.info("%s was unlocked.", str(self)) | ||||||
|  |  | ||||||
|     @property |     @property | ||||||
|     def num_running(self) -> int: |     def num_running(self) -> int: | ||||||
| @@ -187,7 +233,7 @@ class BaseTaskPool: | |||||||
|         finally: |         finally: | ||||||
|             await self._task_ending(task_id, custom_callback=end_callback) |             await self._task_ending(task_id, custom_callback=end_callback) | ||||||
|  |  | ||||||
|     async def _start_task(self, awaitable: Awaitable, ignore_closed: bool = False, end_callback: EndCallbackT = None, |     async def _start_task(self, awaitable: Awaitable, ignore_lock: bool = False, end_callback: EndCallbackT = None, | ||||||
|                           cancel_callback: CancelCallbackT = None) -> int: |                           cancel_callback: CancelCallbackT = None) -> int: | ||||||
|         """ |         """ | ||||||
|         Starts a coroutine as a new task in the pool. |         Starts a coroutine as a new task in the pool. | ||||||
| @@ -197,8 +243,8 @@ class BaseTaskPool: | |||||||
|         Args: |         Args: | ||||||
|             awaitable: |             awaitable: | ||||||
|                 The actual coroutine to be run within the task pool. |                 The actual coroutine to be run within the task pool. | ||||||
|             ignore_closed (optional): |             ignore_lock (optional): | ||||||
|                 If `True`, even if the pool is closed, the task will still be started. |                 If `True`, even if the pool is locked, the task will still be started. | ||||||
|             end_callback (optional): |             end_callback (optional): | ||||||
|                 A callback to execute after the task has ended. |                 A callback to execute after the task has ended. | ||||||
|                 It is run with the task's ID as its only positional argument. |                 It is run with the task's ID as its only positional argument. | ||||||
| @@ -208,12 +254,12 @@ class BaseTaskPool: | |||||||
|  |  | ||||||
|         Raises: |         Raises: | ||||||
|             `asyncio_taskpool.exceptions.NotCoroutine` if `awaitable` is not a coroutine. |             `asyncio_taskpool.exceptions.NotCoroutine` if `awaitable` is not a coroutine. | ||||||
|             `asyncio_taskpool.exceptions.PoolIsClosed` if the pool has been closed and `ignore_closed` is `False`. |             `asyncio_taskpool.exceptions.PoolIsLocked` if the pool has been locked and `ignore_lock` is `False`. | ||||||
|         """ |         """ | ||||||
|         if not iscoroutine(awaitable): |         if not iscoroutine(awaitable): | ||||||
|             raise exceptions.NotCoroutine(f"Not awaitable: {awaitable}") |             raise exceptions.NotCoroutine(f"Not awaitable: {awaitable}") | ||||||
|         if not (self.is_open or ignore_closed): |         if self._locked and not ignore_lock: | ||||||
|             raise exceptions.PoolIsClosed("Cannot start new tasks") |             raise exceptions.PoolIsLocked("Cannot start new tasks") | ||||||
|         await self._enough_room.acquire() |         await self._enough_room.acquire() | ||||||
|         task_id = self._counter |         task_id = self._counter | ||||||
|         self._counter += 1 |         self._counter += 1 | ||||||
| @@ -303,16 +349,11 @@ class BaseTaskPool: | |||||||
|             self._interrupt_flag.clear() |             self._interrupt_flag.clear() | ||||||
|         return results |         return results | ||||||
|  |  | ||||||
|     def close(self) -> None: |  | ||||||
|         """Disallows any more tasks to be started in the pool.""" |  | ||||||
|         self._open = False |  | ||||||
|         log.info("%s is closed!", str(self)) |  | ||||||
|  |  | ||||||
|     async def gather(self, return_exceptions: bool = False): |     async def gather(self, return_exceptions: bool = False): | ||||||
|         """ |         """ | ||||||
|         Calls `asyncio.gather` on **all** tasks from the pool, returns their results, and forgets the tasks. |         Calls `asyncio.gather` on **all** tasks from the pool, returns their results, and forgets the tasks. | ||||||
|  |  | ||||||
|         The `close()` method must have been called prior to this. |         The `lock()` method must have been called prior to this. | ||||||
|  |  | ||||||
|         Note that there may be an unknown number of coroutine functions "queued" to be run as tasks. |         Note that there may be an unknown number of coroutine functions "queued" to be run as tasks. | ||||||
|         This can happen, if for example the `TaskPool.map` method was called with `num_tasks` set to a number smaller |         This can happen, if for example the `TaskPool.map` method was called with `num_tasks` set to a number smaller | ||||||
| @@ -327,10 +368,10 @@ class BaseTaskPool: | |||||||
|             return_exceptions (optional): Passed directly into `gather`. |             return_exceptions (optional): Passed directly into `gather`. | ||||||
|  |  | ||||||
|         Raises: |         Raises: | ||||||
|             `asyncio_taskpool.exceptions.PoolStillOpen` if the pool has not been closed yet. |             `asyncio_taskpool.exceptions.PoolStillUnlocked` if the pool has not been locked yet. | ||||||
|         """ |         """ | ||||||
|         if self._open: |         if not self._locked: | ||||||
|             raise exceptions.PoolStillOpen("Pool must be closed, before tasks can be gathered") |             raise exceptions.PoolStillUnlocked("Pool must be locked, before tasks can be gathered") | ||||||
|         await gather(*self._before_gathering) |         await gather(*self._before_gathering) | ||||||
|         results = await gather(*self._ended.values(), *self._cancelled.values(), *self._running.values(), |         results = await gather(*self._ended.values(), *self._cancelled.values(), *self._running.values(), | ||||||
|                                return_exceptions=return_exceptions) |                                return_exceptions=return_exceptions) | ||||||
| @@ -415,7 +456,7 @@ class TaskPool(BaseTaskPool): | |||||||
|  |  | ||||||
|         Raises: |         Raises: | ||||||
|             `NotCoroutine` if `func` is not a coroutine function. |             `NotCoroutine` if `func` is not a coroutine function. | ||||||
|             `PoolIsClosed` if the pool has been closed already. |             `PoolIsLocked` if the pool has been locked already. | ||||||
|         """ |         """ | ||||||
|         ids = await gather(*(self._apply_one(func, args, kwargs, end_callback, cancel_callback) for _ in range(num))) |         ids = await gather(*(self._apply_one(func, args, kwargs, end_callback, cancel_callback) for _ in range(num))) | ||||||
|         # TODO: for some reason PyCharm wrongly claims that `gather` returns a tuple of exceptions |         # TODO: for some reason PyCharm wrongly claims that `gather` returns a tuple of exceptions | ||||||
| @@ -468,7 +509,7 @@ class TaskPool(BaseTaskPool): | |||||||
|         try: |         try: | ||||||
|             await self._start_task( |             await self._start_task( | ||||||
|                 star_function(func, arg, arg_stars=arg_stars), |                 star_function(func, arg, arg_stars=arg_stars), | ||||||
|                 ignore_closed=True, |                 ignore_lock=True, | ||||||
|                 end_callback=partial(TaskPool._queue_callback, self, q=q, first_batch_started=first_batch_started, |                 end_callback=partial(TaskPool._queue_callback, self, q=q, first_batch_started=first_batch_started, | ||||||
|                                      func=func, arg_stars=arg_stars, end_callback=end_callback, |                                      func=func, arg_stars=arg_stars, end_callback=end_callback, | ||||||
|                                      cancel_callback=cancel_callback), |                                      cancel_callback=cancel_callback), | ||||||
| @@ -579,10 +620,10 @@ class TaskPool(BaseTaskPool): | |||||||
|                 It is run with the task's ID as its only positional argument. |                 It is run with the task's ID as its only positional argument. | ||||||
|  |  | ||||||
|         Raises: |         Raises: | ||||||
|             `asyncio_taskpool.exceptions.PoolIsClosed` if the pool has been closed. |             `asyncio_taskpool.exceptions.PoolIsLocked` if the pool has been locked. | ||||||
|         """ |         """ | ||||||
|         if not self.is_open: |         if not self._locked: | ||||||
|             raise exceptions.PoolIsClosed("Cannot start new tasks") |             raise exceptions.PoolIsLocked("Cannot start new tasks") | ||||||
|         args_queue = self._set_up_args_queue(args_iter, num_tasks) |         args_queue = self._set_up_args_queue(args_iter, num_tasks) | ||||||
|         # We need a flag to ensure that starting all tasks from the first batch here will not be blocked by the |         # We need a flag to ensure that starting all tasks from the first batch here will not be blocked by the | ||||||
|         # `_queue_callback` triggered by one or more of them. |         # `_queue_callback` triggered by one or more of them. | ||||||
| @@ -628,7 +669,7 @@ class TaskPool(BaseTaskPool): | |||||||
|                 It is run with the task's ID as its only positional argument. |                 It is run with the task's ID as its only positional argument. | ||||||
|  |  | ||||||
|         Raises: |         Raises: | ||||||
|             `PoolIsClosed` if the pool has been closed. |             `PoolIsLocked` if the pool has been locked. | ||||||
|             `NotCoroutine` if `func` is not a coroutine function. |             `NotCoroutine` if `func` is not a coroutine function. | ||||||
|         """ |         """ | ||||||
|         await self._map(func, arg_iter, arg_stars=0, num_tasks=num_tasks, |         await self._map(func, arg_iter, arg_stars=0, num_tasks=num_tasks, | ||||||
|   | |||||||
| @@ -1,3 +1,24 @@ | |||||||
|  | __author__ = "Daniil Fajnberg" | ||||||
|  | __copyright__ = "Copyright © 2022 Daniil Fajnberg" | ||||||
|  | __license__ = """GNU LGPLv3.0 | ||||||
|  |  | ||||||
|  | This file is part of asyncio-taskpool. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is free software: you can redistribute it and/or modify it under the terms of | ||||||
|  | version 3.0 of the GNU Lesser General Public License as published by the Free Software Foundation. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;  | ||||||
|  | without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  | ||||||
|  | See the GNU Lesser General Public License for more details. | ||||||
|  |  | ||||||
|  | You should have received a copy of the GNU Lesser General Public License along with asyncio-taskpool.  | ||||||
|  | If not, see <https://www.gnu.org/licenses/>.""" | ||||||
|  |  | ||||||
|  | __doc__ = """ | ||||||
|  | This module contains the task pool control server class definitions. | ||||||
|  | """ | ||||||
|  |  | ||||||
|  |  | ||||||
| import logging | import logging | ||||||
| from abc import ABC, abstractmethod | from abc import ABC, abstractmethod | ||||||
| from asyncio import AbstractServer | from asyncio import AbstractServer | ||||||
| @@ -5,126 +26,126 @@ from asyncio.exceptions import CancelledError | |||||||
| from asyncio.streams import StreamReader, StreamWriter, start_unix_server | from asyncio.streams import StreamReader, StreamWriter, start_unix_server | ||||||
| from asyncio.tasks import Task, create_task | from asyncio.tasks import Task, create_task | ||||||
| from pathlib import Path | from pathlib import Path | ||||||
| from typing import Tuple, Union, Optional | from typing import Optional, Union | ||||||
|  |  | ||||||
| from . import constants |  | ||||||
| from .pool import SimpleTaskPool |  | ||||||
| from .client import ControlClient, UnixControlClient | from .client import ControlClient, UnixControlClient | ||||||
|  | from .pool import TaskPool, SimpleTaskPool | ||||||
|  | from .session import ControlSession | ||||||
|  | from .types import ConnectedCallbackT | ||||||
|  |  | ||||||
|  |  | ||||||
| log = logging.getLogger(__name__) | log = logging.getLogger(__name__) | ||||||
|  |  | ||||||
|  |  | ||||||
| def tasks_str(num: int) -> str: |  | ||||||
|     return "tasks" if num != 1 else "task" |  | ||||||
|  |  | ||||||
|  |  | ||||||
| def get_cmd_arg(msg: str) -> Union[Tuple[str, Optional[int]], Tuple[None, None]]: |  | ||||||
|     cmd = msg.strip().split(' ', 1) |  | ||||||
|     if len(cmd) > 1: |  | ||||||
|         try: |  | ||||||
|             return cmd[0], int(cmd[1]) |  | ||||||
|         except ValueError: |  | ||||||
|             return None, None |  | ||||||
|     return cmd[0], None |  | ||||||
|  |  | ||||||
|  |  | ||||||
| class ControlServer(ABC):  # TODO: Implement interface for normal TaskPool instances, not just SimpleTaskPool | class ControlServer(ABC):  # TODO: Implement interface for normal TaskPool instances, not just SimpleTaskPool | ||||||
|     client_class = ControlClient |     """ | ||||||
|  |     Abstract base class for a task pool control server. | ||||||
|  |  | ||||||
|  |     This class acts as a wrapper around an async server instance and initializes a `ControlSession` upon a client | ||||||
|  |     connecting to it. The entire interface is defined within that session class. | ||||||
|  |     """ | ||||||
|  |     _client_class = ControlClient | ||||||
|  |  | ||||||
|  |     @classmethod | ||||||
|  |     @property | ||||||
|  |     def client_class_name(cls) -> str: | ||||||
|  |         """Returns the name of the control client class matching the server class.""" | ||||||
|  |         return cls._client_class.__name__ | ||||||
|  |  | ||||||
|     @abstractmethod |     @abstractmethod | ||||||
|     async def get_server_instance(self, client_connected_cb, **kwargs) -> AbstractServer: |     async def _get_server_instance(self, client_connected_cb: ConnectedCallbackT, **kwargs) -> AbstractServer: | ||||||
|  |         """ | ||||||
|  |         Initializes, starts, and returns an async server instance (Unix or TCP type). | ||||||
|  |  | ||||||
|  |         Args: | ||||||
|  |             client_connected_cb: | ||||||
|  |                 The callback for when a client connects to the server (as per `asyncio.start_server` or | ||||||
|  |                 `asyncio.start_unix_server`); will always be the internal `_client_connected_cb` method. | ||||||
|  |             **kwargs (optional): | ||||||
|  |                 Keyword arguments to pass into the function that starts the server. | ||||||
|  |  | ||||||
|  |         Returns: | ||||||
|  |             The running server object (a type of `asyncio.Server`). | ||||||
|  |         """ | ||||||
|         raise NotImplementedError |         raise NotImplementedError | ||||||
|  |  | ||||||
|     @abstractmethod |     @abstractmethod | ||||||
|     def final_callback(self) -> None: |     def _final_callback(self) -> None: | ||||||
|  |         """The method to run after the server's `serve_forever` methods ends for whatever reason.""" | ||||||
|         raise NotImplementedError |         raise NotImplementedError | ||||||
|  |  | ||||||
|     def __init__(self, pool: SimpleTaskPool, **server_kwargs) -> None: |     def __init__(self, pool: Union[TaskPool, SimpleTaskPool], **server_kwargs) -> None: | ||||||
|         self._pool: SimpleTaskPool = pool |         """ | ||||||
|  |         Initializes by merely saving the internal attributes, but without starting the server yet. | ||||||
|  |         The task pool must be passed here and can not be set/changed afterwards. This means a control server is always | ||||||
|  |         tied to one specific task pool. | ||||||
|  |  | ||||||
|  |         Args: | ||||||
|  |             pool: | ||||||
|  |                 An instance of a `BaseTaskPool` subclass to tie the server to. | ||||||
|  |             **server_kwargs (optional): | ||||||
|  |                 Keyword arguments that will be passed into the function that starts the server. | ||||||
|  |         """ | ||||||
|  |         self._pool: Union[TaskPool, SimpleTaskPool] = pool | ||||||
|         self._server_kwargs = server_kwargs |         self._server_kwargs = server_kwargs | ||||||
|         self._server: Optional[AbstractServer] = None |         self._server: Optional[AbstractServer] = None | ||||||
|  |  | ||||||
|     async def _start_tasks(self, writer: StreamWriter, num: int = None) -> None: |     @property | ||||||
|         if num is None: |     def pool(self) -> Union[TaskPool, SimpleTaskPool]: | ||||||
|             num = 1 |         """Read-only property for accessing the task pool instance controlled by the server.""" | ||||||
|         log.debug("%s requests starting %s %s", self.client_class.__name__, num, tasks_str(num)) |         return self._pool | ||||||
|         writer.write(str(await self._pool.start(num)).encode()) |  | ||||||
|  |  | ||||||
|     def _stop_tasks(self, writer: StreamWriter, num: int = None) -> None: |     def is_serving(self) -> bool: | ||||||
|         if num is None: |         """Wrapper around the `asyncio.Server.is_serving` method.""" | ||||||
|             num = 1 |         return self._server.is_serving() | ||||||
|         log.debug("%s requests stopping %s %s", self.client_class.__name__, num, tasks_str(num)) |  | ||||||
|         # the requested number may be greater than the total number of running tasks |  | ||||||
|         writer.write(str(self._pool.stop(num)).encode()) |  | ||||||
|  |  | ||||||
|     def _stop_all_tasks(self, writer: StreamWriter) -> None: |  | ||||||
|         log.debug("%s requests stopping all tasks", self.client_class.__name__) |  | ||||||
|         writer.write(str(self._pool.stop_all()).encode()) |  | ||||||
|  |  | ||||||
|     def _pool_size(self, writer: StreamWriter) -> None: |  | ||||||
|         log.debug("%s requests number of running tasks", self.client_class.__name__) |  | ||||||
|         writer.write(str(self._pool.num_running).encode()) |  | ||||||
|  |  | ||||||
|     def _pool_func(self, writer: StreamWriter) -> None: |  | ||||||
|         log.debug("%s requests pool function", self.client_class.__name__) |  | ||||||
|         writer.write(self._pool.func_name.encode()) |  | ||||||
|  |  | ||||||
|     async def _listen(self, reader: StreamReader, writer: StreamWriter) -> None: |  | ||||||
|         while self._server.is_serving(): |  | ||||||
|             msg = (await reader.read(constants.MSG_BYTES)).decode().strip() |  | ||||||
|             if not msg: |  | ||||||
|                 log.debug("%s disconnected", self.client_class.__name__) |  | ||||||
|                 break |  | ||||||
|             cmd, arg = get_cmd_arg(msg) |  | ||||||
|             if cmd == constants.CMD_START: |  | ||||||
|                 await self._start_tasks(writer, arg) |  | ||||||
|             elif cmd == constants.CMD_STOP: |  | ||||||
|                 self._stop_tasks(writer, arg) |  | ||||||
|             elif cmd == constants.CMD_STOP_ALL: |  | ||||||
|                 self._stop_all_tasks(writer) |  | ||||||
|             elif cmd == constants.CMD_NUM_RUNNING: |  | ||||||
|                 self._pool_size(writer) |  | ||||||
|             elif cmd == constants.CMD_FUNC: |  | ||||||
|                 self._pool_func(writer) |  | ||||||
|             else: |  | ||||||
|                 log.debug("%s sent invalid command: %s", self.client_class.__name__, msg) |  | ||||||
|                 writer.write(b"Invalid command!") |  | ||||||
|             await writer.drain() |  | ||||||
|  |  | ||||||
|     async def _client_connected_cb(self, reader: StreamReader, writer: StreamWriter) -> None: |     async def _client_connected_cb(self, reader: StreamReader, writer: StreamWriter) -> None: | ||||||
|         log.debug("%s connected", self.client_class.__name__) |         """ | ||||||
|         writer.write(str(self._pool).encode()) |         The universal client callback that will be passed into the `_get_server_instance` method. | ||||||
|         await writer.drain() |         Instantiates a control session, performs the client handshake, and enters the session's `listen` loop. | ||||||
|         await self._listen(reader, writer) |         """ | ||||||
|  |         session = ControlSession(self, reader, writer) | ||||||
|  |         await session.client_handshake() | ||||||
|  |         await session.listen() | ||||||
|  |  | ||||||
|     async def _serve_forever(self) -> None: |     async def _serve_forever(self) -> None: | ||||||
|  |         """ | ||||||
|  |         To be run as an `asyncio.Task` by the following method. | ||||||
|  |         Serves as a wrapper around the the `asyncio.Server.serve_forever` method that ensures that the `_final_callback` | ||||||
|  |         method is called, when the former method ends for whatever reason. | ||||||
|  |         """ | ||||||
|         try: |         try: | ||||||
|             async with self._server: |             async with self._server: | ||||||
|                 await self._server.serve_forever() |                 await self._server.serve_forever() | ||||||
|         except CancelledError: |         except CancelledError: | ||||||
|             log.debug("%s stopped", self.__class__.__name__) |             log.debug("%s stopped", self.__class__.__name__) | ||||||
|         finally: |         finally: | ||||||
|             self.final_callback() |             self._final_callback() | ||||||
|  |  | ||||||
|     async def serve_forever(self) -> Task: |     async def serve_forever(self) -> Task: | ||||||
|  |         """ | ||||||
|  |         This method actually starts the server and begins listening to client connections on the specified interface. | ||||||
|  |         It should never block because the serving will be performed in a separate task. | ||||||
|  |         """ | ||||||
|         log.debug("Starting %s...", self.__class__.__name__) |         log.debug("Starting %s...", self.__class__.__name__) | ||||||
|         self._server = await self.get_server_instance(self._client_connected_cb, **self._server_kwargs) |         self._server = await self._get_server_instance(self._client_connected_cb, **self._server_kwargs) | ||||||
|         return create_task(self._serve_forever()) |         return create_task(self._serve_forever()) | ||||||
|  |  | ||||||
|  |  | ||||||
| class UnixControlServer(ControlServer): | class UnixControlServer(ControlServer): | ||||||
|     client_class = UnixControlClient |     """Task pool control server class that exposes a unix socket for control clients to connect to.""" | ||||||
|  |     _client_class = UnixControlClient | ||||||
|  |  | ||||||
|     def __init__(self, pool: SimpleTaskPool, **server_kwargs) -> None: |     def __init__(self, pool: SimpleTaskPool, **server_kwargs) -> None: | ||||||
|         self._socket_path = Path(server_kwargs.pop('path')) |         self._socket_path = Path(server_kwargs.pop('path')) | ||||||
|         super().__init__(pool, **server_kwargs) |         super().__init__(pool, **server_kwargs) | ||||||
|  |  | ||||||
|     async def get_server_instance(self, client_connected_cb, **kwargs) -> AbstractServer: |     async def _get_server_instance(self, client_connected_cb: ConnectedCallbackT, **kwargs) -> AbstractServer: | ||||||
|         srv = await start_unix_server(client_connected_cb, self._socket_path, **kwargs) |         server = await start_unix_server(client_connected_cb, self._socket_path, **kwargs) | ||||||
|         log.debug("Opened socket '%s'", str(self._socket_path)) |         log.debug("Opened socket '%s'", str(self._socket_path)) | ||||||
|         return srv |         return server | ||||||
|  |  | ||||||
|     def final_callback(self) -> None: |     def _final_callback(self) -> None: | ||||||
|  |         """Removes the unix socket on which the server was listening.""" | ||||||
|         self._socket_path.unlink() |         self._socket_path.unlink() | ||||||
|         log.debug("Removed socket '%s'", str(self._socket_path)) |         log.debug("Removed socket '%s'", str(self._socket_path)) | ||||||
|   | |||||||
							
								
								
									
										157
									
								
								src/asyncio_taskpool/session.py
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										157
									
								
								src/asyncio_taskpool/session.py
									
									
									
									
									
										Normal file
									
								
							| @@ -0,0 +1,157 @@ | |||||||
|  | import logging | ||||||
|  | import json | ||||||
|  | from argparse import ArgumentError, HelpFormatter, Namespace | ||||||
|  | from asyncio.streams import StreamReader, StreamWriter | ||||||
|  | from typing import Callable, Optional, Union, TYPE_CHECKING | ||||||
|  |  | ||||||
|  | from .constants import CMD, SESSION_PARSER_WRITER, SESSION_MSG_BYTES, CLIENT_INFO | ||||||
|  | from .exceptions import HelpRequested | ||||||
|  | from .helpers import get_first_doc_line, return_or_exception, tasks_str | ||||||
|  | from .pool import TaskPool, SimpleTaskPool | ||||||
|  | from .session_parser import CommandParser, NUM | ||||||
|  |  | ||||||
|  | if TYPE_CHECKING: | ||||||
|  |     from .server import ControlServer | ||||||
|  |  | ||||||
|  |  | ||||||
|  | log = logging.getLogger(__name__) | ||||||
|  |  | ||||||
|  |  | ||||||
|  | class ControlSession: | ||||||
|  |     def __init__(self, server: 'ControlServer', reader: StreamReader, writer: StreamWriter) -> None: | ||||||
|  |         self._control_server: 'ControlServer' = server | ||||||
|  |         self._pool: Union[TaskPool, SimpleTaskPool] = server.pool | ||||||
|  |         self._client_class_name = server.client_class_name | ||||||
|  |         self._reader: StreamReader = reader | ||||||
|  |         self._writer: StreamWriter = writer | ||||||
|  |         self._parser: Optional[CommandParser] = None | ||||||
|  |         self._subparsers = None | ||||||
|  |  | ||||||
|  |     def _add_parser_command(self, name: str, prog: str = None, short_help: str = None, long_help: str = None,  | ||||||
|  |                             **kwargs) -> CommandParser: | ||||||
|  |         if prog is None: | ||||||
|  |             prog = name | ||||||
|  |         kwargs.setdefault('help', short_help or long_help) | ||||||
|  |         kwargs.setdefault('description', long_help or short_help) | ||||||
|  |         return self._subparsers.add_parser(name, prog=prog, parent=self._parser, **kwargs) | ||||||
|  |  | ||||||
|  |     def _add_base_commands(self) -> None: | ||||||
|  |         self._subparsers = self._parser.add_subparsers(title="Commands", dest=CMD.CMD) | ||||||
|  |         self._add_parser_command(CMD.NAME, short_help=get_first_doc_line(self._pool.__class__.__str__)) | ||||||
|  |         self._add_parser_command( | ||||||
|  |             CMD.POOL_SIZE,  | ||||||
|  |             short_help="Get/set the maximum number of tasks in the pool.",  | ||||||
|  |             formatter_class=HelpFormatter | ||||||
|  |         ).add_optional_num_argument( | ||||||
|  |             default=None, | ||||||
|  |             help=f"If passed a number: {get_first_doc_line(self._pool.__class__.pool_size.fset)} " | ||||||
|  |                  f"If omitted: {get_first_doc_line(self._pool.__class__.pool_size.fget)}" | ||||||
|  |         ) | ||||||
|  |         self._add_parser_command( | ||||||
|  |             CMD.NUM_RUNNING, short_help=get_first_doc_line(self._pool.__class__.num_running.fget) | ||||||
|  |         ) | ||||||
|  |  | ||||||
|  |     def _add_simple_commands(self) -> None: | ||||||
|  |         self._add_parser_command( | ||||||
|  |             CMD.START, short_help=get_first_doc_line(self._pool.__class__.start) | ||||||
|  |         ).add_optional_num_argument( | ||||||
|  |             help="Number of tasks to start." | ||||||
|  |         ) | ||||||
|  |         self._add_parser_command( | ||||||
|  |             CMD.STOP, short_help=get_first_doc_line(self._pool.__class__.stop) | ||||||
|  |         ).add_optional_num_argument( | ||||||
|  |             help="Number of tasks to stop." | ||||||
|  |         ) | ||||||
|  |         self._add_parser_command(CMD.STOP_ALL, short_help=get_first_doc_line(self._pool.__class__.stop_all)) | ||||||
|  |         self._add_parser_command( | ||||||
|  |             CMD.FUNC_NAME, short_help=get_first_doc_line(self._pool.__class__.func_name.fget) | ||||||
|  |         ) | ||||||
|  |  | ||||||
|  |     def _init_parser(self, client_terminal_width: int) -> None: | ||||||
|  |         parser_kwargs = { | ||||||
|  |             'prog': '', | ||||||
|  |             SESSION_PARSER_WRITER: self._writer, | ||||||
|  |             CLIENT_INFO.TERMINAL_WIDTH: client_terminal_width, | ||||||
|  |         } | ||||||
|  |         self._parser = CommandParser(**parser_kwargs) | ||||||
|  |         self._add_base_commands() | ||||||
|  |         if isinstance(self._pool, TaskPool): | ||||||
|  |             pass  # TODO | ||||||
|  |         elif isinstance(self._pool, SimpleTaskPool): | ||||||
|  |             self._add_simple_commands() | ||||||
|  |  | ||||||
|  |     async def client_handshake(self) -> None: | ||||||
|  |         client_info = json.loads((await self._reader.read(SESSION_MSG_BYTES)).decode().strip()) | ||||||
|  |         log.debug("%s connected", self._client_class_name) | ||||||
|  |         self._init_parser(client_info[CLIENT_INFO.TERMINAL_WIDTH]) | ||||||
|  |         self._writer.write(str(self._pool).encode()) | ||||||
|  |         await self._writer.drain() | ||||||
|  |  | ||||||
|  |     async def _write_function_output(self, func: Callable, *args, **kwargs) -> None: | ||||||
|  |         output = await return_or_exception(func, *args, **kwargs) | ||||||
|  |         self._writer.write(b"ok" if output is None else str(output).encode()) | ||||||
|  |  | ||||||
|  |     async def _cmd_name(self, **_kwargs) -> None: | ||||||
|  |         log.debug("%s requests task pool name", self._client_class_name) | ||||||
|  |         await self._write_function_output(self._pool.__class__.__str__, self._pool) | ||||||
|  |  | ||||||
|  |     async def _cmd_pool_size(self, **kwargs) -> None: | ||||||
|  |         num = kwargs.get(NUM) | ||||||
|  |         if num is None: | ||||||
|  |             log.debug("%s requests pool size", self._client_class_name) | ||||||
|  |             await self._write_function_output(self._pool.__class__.pool_size.fget, self._pool) | ||||||
|  |         else: | ||||||
|  |             log.debug("%s requests setting pool size to %s", self._client_class_name, num) | ||||||
|  |             await self._write_function_output(self._pool.__class__.pool_size.fset, self._pool, num) | ||||||
|  |  | ||||||
|  |     async def _cmd_num_running(self, **_kwargs) -> None: | ||||||
|  |         log.debug("%s requests number of running tasks", self._client_class_name) | ||||||
|  |         await self._write_function_output(self._pool.__class__.num_running.fget, self._pool) | ||||||
|  |  | ||||||
|  |     async def _cmd_start(self, **kwargs) -> None: | ||||||
|  |         num = kwargs[NUM] | ||||||
|  |         log.debug("%s requests starting %s %s", self._client_class_name, num, tasks_str(num)) | ||||||
|  |         await self._write_function_output(self._pool.start, num) | ||||||
|  |  | ||||||
|  |     async def _cmd_stop(self, **kwargs) -> None: | ||||||
|  |         num = kwargs[NUM] | ||||||
|  |         log.debug("%s requests stopping %s %s", self._client_class_name, num, tasks_str(num)) | ||||||
|  |         await self._write_function_output(self._pool.stop, num) | ||||||
|  |  | ||||||
|  |     async def _cmd_stop_all(self, **_kwargs) -> None: | ||||||
|  |         log.debug("%s requests stopping all tasks", self._client_class_name) | ||||||
|  |         await self._write_function_output(self._pool.stop_all) | ||||||
|  |  | ||||||
|  |     async def _cmd_func_name(self, **_kwargs) -> None: | ||||||
|  |         log.debug("%s requests pool function name", self._client_class_name) | ||||||
|  |         await self._write_function_output(self._pool.__class__.func_name.fget, self._pool) | ||||||
|  |  | ||||||
|  |     async def _execute_command(self, args: Namespace) -> None: | ||||||
|  |         args = vars(args) | ||||||
|  |         cmd: str = args.pop(CMD.CMD, None) | ||||||
|  |         if cmd is not None: | ||||||
|  |             method = getattr(self, f'_cmd_{cmd.replace("-", "_")}') | ||||||
|  |             await method(**args) | ||||||
|  |  | ||||||
|  |     async def _parse_command(self, msg: str) -> None: | ||||||
|  |         try: | ||||||
|  |             args, argv = self._parser.parse_known_args(msg.split(' ')) | ||||||
|  |         except ArgumentError as e: | ||||||
|  |             self._writer.write(str(e).encode()) | ||||||
|  |             return | ||||||
|  |         except HelpRequested: | ||||||
|  |             return | ||||||
|  |         if argv: | ||||||
|  |             log.debug("%s sent unknown arguments: %s", self._client_class_name, msg) | ||||||
|  |             self._writer.write(b"Invalid command!") | ||||||
|  |             return | ||||||
|  |         await self._execute_command(args) | ||||||
|  |  | ||||||
|  |     async def listen(self) -> None: | ||||||
|  |         while self._control_server.is_serving(): | ||||||
|  |             msg = (await self._reader.read(SESSION_MSG_BYTES)).decode().strip() | ||||||
|  |             if not msg: | ||||||
|  |                 log.debug("%s disconnected", self._client_class_name) | ||||||
|  |                 break | ||||||
|  |             await self._parse_command(msg) | ||||||
|  |             await self._writer.drain() | ||||||
							
								
								
									
										60
									
								
								src/asyncio_taskpool/session_parser.py
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										60
									
								
								src/asyncio_taskpool/session_parser.py
									
									
									
									
									
										Normal file
									
								
							| @@ -0,0 +1,60 @@ | |||||||
|  | from argparse import Action, ArgumentParser, ArgumentDefaultsHelpFormatter, HelpFormatter | ||||||
|  | from asyncio.streams import StreamWriter | ||||||
|  | from typing import Type, TypeVar | ||||||
|  |  | ||||||
|  | from .constants import SESSION_PARSER_WRITER, CLIENT_INFO | ||||||
|  | from .exceptions import HelpRequested | ||||||
|  |  | ||||||
|  |  | ||||||
|  | FmtCls = TypeVar('FmtCls', bound=Type[HelpFormatter]) | ||||||
|  | FORMATTER_CLASS = 'formatter_class' | ||||||
|  | NUM = 'num' | ||||||
|  |  | ||||||
|  |  | ||||||
|  | class CommandParser(ArgumentParser): | ||||||
|  |     @staticmethod | ||||||
|  |     def help_formatter_factory(terminal_width: int, base_cls: FmtCls = None) -> FmtCls: | ||||||
|  |         if base_cls is None: | ||||||
|  |             base_cls = ArgumentDefaultsHelpFormatter | ||||||
|  |  | ||||||
|  |         class ClientHelpFormatter(base_cls): | ||||||
|  |             def __init__(self, *args, **kwargs) -> None: | ||||||
|  |                 kwargs['width'] = terminal_width | ||||||
|  |                 super().__init__(*args, **kwargs) | ||||||
|  |         return ClientHelpFormatter | ||||||
|  |  | ||||||
|  |     def __init__(self, *args, **kwargs) -> None: | ||||||
|  |         parent: CommandParser = kwargs.pop('parent', None) | ||||||
|  |         self._stream_writer: StreamWriter = parent.stream_writer if parent else kwargs.pop(SESSION_PARSER_WRITER) | ||||||
|  |         self._terminal_width: int = parent.terminal_width if parent else kwargs.pop(CLIENT_INFO.TERMINAL_WIDTH) | ||||||
|  |         kwargs[FORMATTER_CLASS] = self.help_formatter_factory(self._terminal_width, kwargs.get(FORMATTER_CLASS)) | ||||||
|  |         kwargs.setdefault('exit_on_error', False) | ||||||
|  |         super().__init__(*args, **kwargs) | ||||||
|  |  | ||||||
|  |     @property | ||||||
|  |     def stream_writer(self) -> StreamWriter: | ||||||
|  |         return self._stream_writer | ||||||
|  |  | ||||||
|  |     @property | ||||||
|  |     def terminal_width(self) -> int: | ||||||
|  |         return self._terminal_width | ||||||
|  |  | ||||||
|  |     def _print_message(self, message: str, *args, **kwargs) -> None: | ||||||
|  |         if message: | ||||||
|  |             self.stream_writer.write(message.encode()) | ||||||
|  |  | ||||||
|  |     def exit(self, status: int = 0, message: str = None) -> None: | ||||||
|  |         if message: | ||||||
|  |             self._print_message(message) | ||||||
|  |  | ||||||
|  |     def print_help(self, file=None) -> None: | ||||||
|  |         super().print_help(file) | ||||||
|  |         raise HelpRequested | ||||||
|  |  | ||||||
|  |     def add_optional_num_argument(self, *name_or_flags: str, **kwargs) -> Action: | ||||||
|  |         if not name_or_flags: | ||||||
|  |             name_or_flags = (NUM, ) | ||||||
|  |         kwargs.setdefault('nargs', '?') | ||||||
|  |         kwargs.setdefault('default', 1) | ||||||
|  |         kwargs.setdefault('type', int) | ||||||
|  |         return self.add_argument(*name_or_flags, **kwargs) | ||||||
| @@ -1,3 +1,24 @@ | |||||||
|  | __author__ = "Daniil Fajnberg" | ||||||
|  | __copyright__ = "Copyright © 2022 Daniil Fajnberg" | ||||||
|  | __license__ = """GNU LGPLv3.0 | ||||||
|  |  | ||||||
|  | This file is part of asyncio-taskpool. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is free software: you can redistribute it and/or modify it under the terms of | ||||||
|  | version 3.0 of the GNU Lesser General Public License as published by the Free Software Foundation. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;  | ||||||
|  | without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  | ||||||
|  | See the GNU Lesser General Public License for more details. | ||||||
|  |  | ||||||
|  | You should have received a copy of the GNU Lesser General Public License along with asyncio-taskpool.  | ||||||
|  | If not, see <https://www.gnu.org/licenses/>.""" | ||||||
|  |  | ||||||
|  | __doc__ = """ | ||||||
|  | Custom type definitions used in various modules. | ||||||
|  | """ | ||||||
|  |  | ||||||
|  |  | ||||||
| from asyncio.streams import StreamReader, StreamWriter | from asyncio.streams import StreamReader, StreamWriter | ||||||
| from typing import Any, Awaitable, Callable, Iterable, Mapping, Tuple, TypeVar, Union | from typing import Any, Awaitable, Callable, Iterable, Mapping, Tuple, TypeVar, Union | ||||||
|  |  | ||||||
| @@ -7,10 +28,11 @@ T = TypeVar('T') | |||||||
| ArgsT = Iterable[Any] | ArgsT = Iterable[Any] | ||||||
| KwArgsT = Mapping[str, Any] | KwArgsT = Mapping[str, Any] | ||||||
|  |  | ||||||
| AnyCallableT = Callable[[...], Union[Awaitable[T], T]] | AnyCallableT = Callable[[...], Union[T, Awaitable[T]]] | ||||||
| CoroutineFunc = Callable[[...], Awaitable[Any]] | CoroutineFunc = Callable[[...], Awaitable[Any]] | ||||||
|  |  | ||||||
| EndCallbackT = Callable | EndCallbackT = Callable | ||||||
| CancelCallbackT = Callable | CancelCallbackT = Callable | ||||||
|  |  | ||||||
|  | ConnectedCallbackT = Callable[[StreamReader, StreamWriter], Awaitable[None]] | ||||||
| ClientConnT = Union[Tuple[StreamReader, StreamWriter], Tuple[None, None]] | ClientConnT = Union[Tuple[StreamReader, StreamWriter], Tuple[None, None]] | ||||||
|   | |||||||
							
								
								
									
										96
									
								
								tests/test_helpers.py
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										96
									
								
								tests/test_helpers.py
									
									
									
									
									
										Normal file
									
								
							| @@ -0,0 +1,96 @@ | |||||||
|  | __author__ = "Daniil Fajnberg" | ||||||
|  | __copyright__ = "Copyright © 2022 Daniil Fajnberg" | ||||||
|  | __license__ = """GNU LGPLv3.0 | ||||||
|  |  | ||||||
|  | This file is part of asyncio-taskpool. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is free software: you can redistribute it and/or modify it under the terms of | ||||||
|  | version 3.0 of the GNU Lesser General Public License as published by the Free Software Foundation. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;  | ||||||
|  | without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  | ||||||
|  | See the GNU Lesser General Public License for more details. | ||||||
|  |  | ||||||
|  | You should have received a copy of the GNU Lesser General Public License along with asyncio-taskpool.  | ||||||
|  | If not, see <https://www.gnu.org/licenses/>.""" | ||||||
|  |  | ||||||
|  | __doc__ = """ | ||||||
|  | Unittests for the `asyncio_taskpool.helpers` module. | ||||||
|  | """ | ||||||
|  |  | ||||||
|  |  | ||||||
|  | from unittest import IsolatedAsyncioTestCase | ||||||
|  | from unittest.mock import MagicMock, AsyncMock, NonCallableMagicMock | ||||||
|  |  | ||||||
|  | from asyncio_taskpool import helpers | ||||||
|  |  | ||||||
|  |  | ||||||
|  | class HelpersTestCase(IsolatedAsyncioTestCase): | ||||||
|  |  | ||||||
|  |     async def test_execute_optional(self): | ||||||
|  |         f, args, kwargs = NonCallableMagicMock(), [1, 2], None | ||||||
|  |         a = [f, args, kwargs]  # to avoid IDE nagging | ||||||
|  |         self.assertIsNone(await helpers.execute_optional(*a)) | ||||||
|  |  | ||||||
|  |         expected_output = 'foo' | ||||||
|  |         f = MagicMock(return_value=expected_output) | ||||||
|  |         output = await helpers.execute_optional(f, args, kwargs) | ||||||
|  |         self.assertEqual(expected_output, output) | ||||||
|  |         f.assert_called_once_with(*args) | ||||||
|  |  | ||||||
|  |         f.reset_mock() | ||||||
|  |  | ||||||
|  |         kwargs = {'a': 100, 'b': 200} | ||||||
|  |         output = await helpers.execute_optional(f, args, kwargs) | ||||||
|  |         self.assertEqual(expected_output, output) | ||||||
|  |         f.assert_called_once_with(*args, **kwargs) | ||||||
|  |  | ||||||
|  |         f = AsyncMock(return_value=expected_output) | ||||||
|  |         output = await helpers.execute_optional(f, args, kwargs) | ||||||
|  |         self.assertEqual(expected_output, output) | ||||||
|  |         f.assert_awaited_once_with(*args, **kwargs) | ||||||
|  |  | ||||||
|  |     def test_star_function(self): | ||||||
|  |         expected_output = 'bar' | ||||||
|  |         f = MagicMock(return_value=expected_output) | ||||||
|  |         a = (1, 2, 3) | ||||||
|  |         stars = 0 | ||||||
|  |         output = helpers.star_function(f, a, stars) | ||||||
|  |         self.assertEqual(expected_output, output) | ||||||
|  |         f.assert_called_once_with(a) | ||||||
|  |  | ||||||
|  |         f.reset_mock() | ||||||
|  |  | ||||||
|  |         stars = 1 | ||||||
|  |         output = helpers.star_function(f, a, stars) | ||||||
|  |         self.assertEqual(expected_output, output) | ||||||
|  |         f.assert_called_once_with(*a) | ||||||
|  |  | ||||||
|  |         f.reset_mock() | ||||||
|  |  | ||||||
|  |         a = {'a': 1, 'b': 2} | ||||||
|  |         stars = 2 | ||||||
|  |         output = helpers.star_function(f, a, stars) | ||||||
|  |         self.assertEqual(expected_output, output) | ||||||
|  |         f.assert_called_once_with(**a) | ||||||
|  |  | ||||||
|  |         with self.assertRaises(ValueError): | ||||||
|  |             helpers.star_function(f, a, 3) | ||||||
|  |         with self.assertRaises(ValueError): | ||||||
|  |             helpers.star_function(f, a, -1) | ||||||
|  |         with self.assertRaises(ValueError): | ||||||
|  |             helpers.star_function(f, a, 123456789) | ||||||
|  |  | ||||||
|  |     async def test_join_queue(self): | ||||||
|  |         mock_join = AsyncMock() | ||||||
|  |         mock_queue = MagicMock(join=mock_join) | ||||||
|  |         self.assertIsNone(await helpers.join_queue(mock_queue)) | ||||||
|  |         mock_join.assert_awaited_once_with() | ||||||
|  |  | ||||||
|  |     def test_task_str(self): | ||||||
|  |         self.assertEqual("task", helpers.tasks_str(1)) | ||||||
|  |         self.assertEqual("tasks", helpers.tasks_str(0)) | ||||||
|  |         self.assertEqual("tasks", helpers.tasks_str(-1)) | ||||||
|  |         self.assertEqual("tasks", helpers.tasks_str(2)) | ||||||
|  |         self.assertEqual("tasks", helpers.tasks_str(-10)) | ||||||
|  |         self.assertEqual("tasks", helpers.tasks_str(42)) | ||||||
| @@ -1,3 +1,24 @@ | |||||||
|  | __author__ = "Daniil Fajnberg" | ||||||
|  | __copyright__ = "Copyright © 2022 Daniil Fajnberg" | ||||||
|  | __license__ = """GNU LGPLv3.0 | ||||||
|  |  | ||||||
|  | This file is part of asyncio-taskpool. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is free software: you can redistribute it and/or modify it under the terms of | ||||||
|  | version 3.0 of the GNU Lesser General Public License as published by the Free Software Foundation. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;  | ||||||
|  | without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  | ||||||
|  | See the GNU Lesser General Public License for more details. | ||||||
|  |  | ||||||
|  | You should have received a copy of the GNU Lesser General Public License along with asyncio-taskpool.  | ||||||
|  | If not, see <https://www.gnu.org/licenses/>.""" | ||||||
|  |  | ||||||
|  | __doc__ = """ | ||||||
|  | Unittests for the `asyncio_taskpool.pool` module. | ||||||
|  | """ | ||||||
|  |  | ||||||
|  |  | ||||||
| import asyncio | import asyncio | ||||||
| from asyncio.exceptions import CancelledError | from asyncio.exceptions import CancelledError | ||||||
| from asyncio.queues import Queue | from asyncio.queues import Queue | ||||||
| @@ -68,7 +89,7 @@ class BaseTaskPoolTestCase(CommonTestCase): | |||||||
|  |  | ||||||
|     def test_init(self): |     def test_init(self): | ||||||
|         self.assertIsInstance(self.task_pool._enough_room, asyncio.locks.Semaphore) |         self.assertIsInstance(self.task_pool._enough_room, asyncio.locks.Semaphore) | ||||||
|         self.assertTrue(self.task_pool._open) |         self.assertFalse(self.task_pool._locked) | ||||||
|         self.assertEqual(0, self.task_pool._counter) |         self.assertEqual(0, self.task_pool._counter) | ||||||
|         self.assertDictEqual(EMPTY_DICT, self.task_pool._running) |         self.assertDictEqual(EMPTY_DICT, self.task_pool._running) | ||||||
|         self.assertDictEqual(EMPTY_DICT, self.task_pool._cancelled) |         self.assertDictEqual(EMPTY_DICT, self.task_pool._cancelled) | ||||||
| @@ -103,9 +124,23 @@ class BaseTaskPoolTestCase(CommonTestCase): | |||||||
|         self.task_pool.pool_size = new_size = 69 |         self.task_pool.pool_size = new_size = 69 | ||||||
|         self.assertEqual(new_size, self.task_pool._pool_size) |         self.assertEqual(new_size, self.task_pool._pool_size) | ||||||
|  |  | ||||||
|     def test_is_open(self): |     def test_is_locked(self): | ||||||
|         self.task_pool._open = FOO |         self.task_pool._locked = FOO | ||||||
|         self.assertEqual(FOO, self.task_pool.is_open) |         self.assertEqual(FOO, self.task_pool.is_locked) | ||||||
|  |  | ||||||
|  |     def test_lock(self): | ||||||
|  |         assert not self.task_pool._locked | ||||||
|  |         self.task_pool.lock() | ||||||
|  |         self.assertTrue(self.task_pool._locked) | ||||||
|  |         self.task_pool.lock() | ||||||
|  |         self.assertTrue(self.task_pool._locked) | ||||||
|  |  | ||||||
|  |     def test_unlock(self): | ||||||
|  |         self.task_pool._locked = True | ||||||
|  |         self.task_pool.unlock() | ||||||
|  |         self.assertFalse(self.task_pool._locked) | ||||||
|  |         self.task_pool.unlock() | ||||||
|  |         self.assertFalse(self.task_pool._locked) | ||||||
|  |  | ||||||
|     def test_num_running(self): |     def test_num_running(self): | ||||||
|         self.task_pool._running = ['foo', 'bar', 'baz'] |         self.task_pool._running = ['foo', 'bar', 'baz'] | ||||||
| @@ -211,11 +246,9 @@ class BaseTaskPoolTestCase(CommonTestCase): | |||||||
|     @patch.object(pool, 'create_task') |     @patch.object(pool, 'create_task') | ||||||
|     @patch.object(pool.BaseTaskPool, '_task_wrapper', new_callable=MagicMock) |     @patch.object(pool.BaseTaskPool, '_task_wrapper', new_callable=MagicMock) | ||||||
|     @patch.object(pool.BaseTaskPool, '_task_name', return_value=FOO) |     @patch.object(pool.BaseTaskPool, '_task_name', return_value=FOO) | ||||||
|     @patch.object(pool.BaseTaskPool, 'is_open', new_callable=PropertyMock) |     async def test__start_task(self, mock__task_name: MagicMock, mock__task_wrapper: AsyncMock, | ||||||
|     async def test__start_task(self, mock_is_open: MagicMock, mock__task_name: MagicMock, |                                mock_create_task: MagicMock): | ||||||
|                                mock__task_wrapper: AsyncMock, mock_create_task: MagicMock): |  | ||||||
|         def reset_mocks() -> None: |         def reset_mocks() -> None: | ||||||
|             mock_is_open.reset_mock() |  | ||||||
|             mock__task_name.reset_mock() |             mock__task_name.reset_mock() | ||||||
|             mock__task_wrapper.reset_mock() |             mock__task_wrapper.reset_mock() | ||||||
|             mock_create_task.reset_mock() |             mock_create_task.reset_mock() | ||||||
| @@ -226,31 +259,27 @@ class BaseTaskPoolTestCase(CommonTestCase): | |||||||
|         self.task_pool._counter = count = 123 |         self.task_pool._counter = count = 123 | ||||||
|         self.task_pool._enough_room._value = room = 123 |         self.task_pool._enough_room._value = room = 123 | ||||||
|  |  | ||||||
|         with self.assertRaises(exceptions.NotCoroutine): |         def check_nothing_changed() -> None: | ||||||
|             await self.task_pool._start_task(MagicMock(), end_callback=mock_end_cb, cancel_callback=mock_cancel_cb) |  | ||||||
|             self.assertEqual(count, self.task_pool._counter) |             self.assertEqual(count, self.task_pool._counter) | ||||||
|             self.assertNotIn(count, self.task_pool._running) |             self.assertNotIn(count, self.task_pool._running) | ||||||
|             self.assertEqual(room, self.task_pool._enough_room._value) |             self.assertEqual(room, self.task_pool._enough_room._value) | ||||||
|         mock_is_open.assert_not_called() |  | ||||||
|             mock__task_name.assert_not_called() |             mock__task_name.assert_not_called() | ||||||
|             mock__task_wrapper.assert_not_called() |             mock__task_wrapper.assert_not_called() | ||||||
|             mock_create_task.assert_not_called() |             mock_create_task.assert_not_called() | ||||||
|             reset_mocks() |             reset_mocks() | ||||||
|  |  | ||||||
|         mock_is_open.return_value = ignore_closed = False |         with self.assertRaises(exceptions.NotCoroutine): | ||||||
|  |             await self.task_pool._start_task(MagicMock(), end_callback=mock_end_cb, cancel_callback=mock_cancel_cb) | ||||||
|  |         check_nothing_changed() | ||||||
|  |  | ||||||
|  |         self.task_pool._locked = True | ||||||
|  |         ignore_closed = False | ||||||
|         mock_awaitable = mock_coroutine() |         mock_awaitable = mock_coroutine() | ||||||
|         with self.assertRaises(exceptions.PoolIsClosed): |         with self.assertRaises(exceptions.PoolIsLocked): | ||||||
|             await self.task_pool._start_task(mock_awaitable, ignore_closed, |             await self.task_pool._start_task(mock_awaitable, ignore_closed, | ||||||
|                                              end_callback=mock_end_cb, cancel_callback=mock_cancel_cb) |                                              end_callback=mock_end_cb, cancel_callback=mock_cancel_cb) | ||||||
|         await mock_awaitable |         await mock_awaitable | ||||||
|         self.assertEqual(count, self.task_pool._counter) |         check_nothing_changed() | ||||||
|         self.assertNotIn(count, self.task_pool._running) |  | ||||||
|         self.assertEqual(room, self.task_pool._enough_room._value) |  | ||||||
|         mock_is_open.assert_called_once_with() |  | ||||||
|         mock__task_name.assert_not_called() |  | ||||||
|         mock__task_wrapper.assert_not_called() |  | ||||||
|         mock_create_task.assert_not_called() |  | ||||||
|         reset_mocks() |  | ||||||
|  |  | ||||||
|         ignore_closed = True |         ignore_closed = True | ||||||
|         mock_awaitable = mock_coroutine() |         mock_awaitable = mock_coroutine() | ||||||
| @@ -261,7 +290,6 @@ class BaseTaskPoolTestCase(CommonTestCase): | |||||||
|         self.assertEqual(count + 1, self.task_pool._counter) |         self.assertEqual(count + 1, self.task_pool._counter) | ||||||
|         self.assertEqual(mock_task, self.task_pool._running[count]) |         self.assertEqual(mock_task, self.task_pool._running[count]) | ||||||
|         self.assertEqual(room - 1, self.task_pool._enough_room._value) |         self.assertEqual(room - 1, self.task_pool._enough_room._value) | ||||||
|         mock_is_open.assert_called_once_with() |  | ||||||
|         mock__task_name.assert_called_once_with(count) |         mock__task_name.assert_called_once_with(count) | ||||||
|         mock__task_wrapper.assert_called_once_with(mock_awaitable, count, mock_end_cb, mock_cancel_cb) |         mock__task_wrapper.assert_called_once_with(mock_awaitable, count, mock_end_cb, mock_cancel_cb) | ||||||
|         mock_create_task.assert_called_once_with(mock_wrapped, name=FOO) |         mock_create_task.assert_called_once_with(mock_wrapped, name=FOO) | ||||||
| @@ -280,7 +308,6 @@ class BaseTaskPoolTestCase(CommonTestCase): | |||||||
|         self.assertEqual(count + 1, self.task_pool._counter) |         self.assertEqual(count + 1, self.task_pool._counter) | ||||||
|         self.assertNotIn(count, self.task_pool._running) |         self.assertNotIn(count, self.task_pool._running) | ||||||
|         self.assertEqual(room, self.task_pool._enough_room._value) |         self.assertEqual(room, self.task_pool._enough_room._value) | ||||||
|         mock_is_open.assert_called_once_with() |  | ||||||
|         mock__task_name.assert_called_once_with(count) |         mock__task_name.assert_called_once_with(count) | ||||||
|         mock__task_wrapper.assert_called_once_with(mock_awaitable, count, mock_end_cb, mock_cancel_cb) |         mock__task_wrapper.assert_called_once_with(mock_awaitable, count, mock_end_cb, mock_cancel_cb) | ||||||
|         mock_create_task.assert_called_once_with(mock_wrapped, name=FOO) |         mock_create_task.assert_called_once_with(mock_wrapped, name=FOO) | ||||||
| @@ -345,11 +372,6 @@ class BaseTaskPoolTestCase(CommonTestCase): | |||||||
|         self.assertDictEqual(self.task_pool._ended, EMPTY_DICT) |         self.assertDictEqual(self.task_pool._ended, EMPTY_DICT) | ||||||
|         self.assertDictEqual(self.task_pool._cancelled, EMPTY_DICT) |         self.assertDictEqual(self.task_pool._cancelled, EMPTY_DICT) | ||||||
|  |  | ||||||
|     def test_close(self): |  | ||||||
|         assert self.task_pool._open |  | ||||||
|         self.task_pool.close() |  | ||||||
|         self.assertFalse(self.task_pool._open) |  | ||||||
|  |  | ||||||
|     async def test_gather(self): |     async def test_gather(self): | ||||||
|         test_exception = TestException() |         test_exception = TestException() | ||||||
|         mock_ended_func, mock_cancelled_func = AsyncMock(return_value=FOO), AsyncMock(side_effect=test_exception) |         mock_ended_func, mock_cancelled_func = AsyncMock(return_value=FOO), AsyncMock(side_effect=test_exception) | ||||||
| @@ -361,8 +383,8 @@ class BaseTaskPoolTestCase(CommonTestCase): | |||||||
|         self.task_pool._running = running = {789: mock_running_func()} |         self.task_pool._running = running = {789: mock_running_func()} | ||||||
|         self.task_pool._interrupt_flag.set() |         self.task_pool._interrupt_flag.set() | ||||||
|  |  | ||||||
|         assert self.task_pool._open |         assert not self.task_pool._locked | ||||||
|         with self.assertRaises(exceptions.PoolStillOpen): |         with self.assertRaises(exceptions.PoolStillUnlocked): | ||||||
|             await self.task_pool.gather() |             await self.task_pool.gather() | ||||||
|         self.assertDictEqual(self.task_pool._ended, ended) |         self.assertDictEqual(self.task_pool._ended, ended) | ||||||
|         self.assertDictEqual(self.task_pool._cancelled, cancelled) |         self.assertDictEqual(self.task_pool._cancelled, cancelled) | ||||||
| @@ -370,7 +392,7 @@ class BaseTaskPoolTestCase(CommonTestCase): | |||||||
|         self.assertListEqual(self.task_pool._before_gathering, before_gather) |         self.assertListEqual(self.task_pool._before_gathering, before_gather) | ||||||
|         self.assertTrue(self.task_pool._interrupt_flag.is_set()) |         self.assertTrue(self.task_pool._interrupt_flag.is_set()) | ||||||
|  |  | ||||||
|         self.task_pool._open = False |         self.task_pool._locked = True | ||||||
|  |  | ||||||
|         def check_assertions(output) -> None: |         def check_assertions(output) -> None: | ||||||
|             self.assertListEqual([FOO, test_exception, BAR], output) |             self.assertListEqual([FOO, test_exception, BAR], output) | ||||||
| @@ -449,7 +471,7 @@ class TaskPoolTestCase(CommonTestCase): | |||||||
|         mock_flag, end_cb, cancel_cb = MagicMock(), MagicMock(), MagicMock() |         mock_flag, end_cb, cancel_cb = MagicMock(), MagicMock(), MagicMock() | ||||||
|         self.assertIsNone(await self.task_pool._queue_consumer(q, mock_flag, mock_func, stars, end_cb, cancel_cb)) |         self.assertIsNone(await self.task_pool._queue_consumer(q, mock_flag, mock_func, stars, end_cb, cancel_cb)) | ||||||
|         self.assertTrue(q.empty()) |         self.assertTrue(q.empty()) | ||||||
|         mock__start_task.assert_awaited_once_with(awaitable, ignore_closed=True, |         mock__start_task.assert_awaited_once_with(awaitable, ignore_lock=True, | ||||||
|                                                   end_callback=queue_callback, cancel_callback=cancel_cb) |                                                   end_callback=queue_callback, cancel_callback=cancel_cb) | ||||||
|         mock_star_function.assert_called_once_with(mock_func, arg, arg_stars=stars) |         mock_star_function.assert_called_once_with(mock_func, arg, arg_stars=stars) | ||||||
|         mock_partial.assert_called_once_with(pool.TaskPool._queue_callback, self.task_pool, |         mock_partial.assert_called_once_with(pool.TaskPool._queue_callback, self.task_pool, | ||||||
| @@ -459,7 +481,7 @@ class TaskPoolTestCase(CommonTestCase): | |||||||
|         mock_star_function.reset_mock() |         mock_star_function.reset_mock() | ||||||
|         mock_partial.reset_mock() |         mock_partial.reset_mock() | ||||||
|  |  | ||||||
|         self.assertIsNone(await self.task_pool._queue_consumer(q, mock_func, stars, end_cb, cancel_cb)) |         self.assertIsNone(await self.task_pool._queue_consumer(q, mock_flag, mock_func, stars, end_cb, cancel_cb)) | ||||||
|         self.assertTrue(q.empty()) |         self.assertTrue(q.empty()) | ||||||
|         mock__start_task.assert_not_awaited() |         mock__start_task.assert_not_awaited() | ||||||
|         mock_star_function.assert_not_called() |         mock_star_function.assert_not_called() | ||||||
| @@ -528,9 +550,8 @@ class TaskPoolTestCase(CommonTestCase): | |||||||
|     @patch.object(pool, 'Event') |     @patch.object(pool, 'Event') | ||||||
|     @patch.object(pool.TaskPool, '_queue_consumer') |     @patch.object(pool.TaskPool, '_queue_consumer') | ||||||
|     @patch.object(pool.TaskPool, '_set_up_args_queue') |     @patch.object(pool.TaskPool, '_set_up_args_queue') | ||||||
|     @patch.object(pool.TaskPool, 'is_open', new_callable=PropertyMock) |     async def test__map(self, mock__set_up_args_queue: MagicMock, mock__queue_consumer: AsyncMock, | ||||||
|     async def test__map(self, mock_is_open: MagicMock, mock__set_up_args_queue: MagicMock, |                         mock_event_cls: MagicMock): | ||||||
|                         mock__queue_consumer: AsyncMock, mock_event_cls: MagicMock): |  | ||||||
|         qsize = 4 |         qsize = 4 | ||||||
|         mock__set_up_args_queue.return_value = mock_q = MagicMock(qsize=MagicMock(return_value=qsize)) |         mock__set_up_args_queue.return_value = mock_q = MagicMock(qsize=MagicMock(return_value=qsize)) | ||||||
|         mock_flag_set = MagicMock() |         mock_flag_set = MagicMock() | ||||||
| @@ -540,17 +561,14 @@ class TaskPoolTestCase(CommonTestCase): | |||||||
|         args_iter, num_tasks = (FOO, BAR, 1, 2, 3), 2 |         args_iter, num_tasks = (FOO, BAR, 1, 2, 3), 2 | ||||||
|         end_cb, cancel_cb = MagicMock(), MagicMock() |         end_cb, cancel_cb = MagicMock(), MagicMock() | ||||||
|  |  | ||||||
|         mock_is_open.return_value = False |         self.task_pool._locked = False | ||||||
|         with self.assertRaises(exceptions.PoolIsClosed): |         with self.assertRaises(exceptions.PoolIsLocked): | ||||||
|             await self.task_pool._map(mock_func, args_iter, stars, num_tasks, end_cb, cancel_cb) |             await self.task_pool._map(mock_func, args_iter, stars, num_tasks, end_cb, cancel_cb) | ||||||
|         mock_is_open.assert_called_once_with() |  | ||||||
|         mock__set_up_args_queue.assert_not_called() |         mock__set_up_args_queue.assert_not_called() | ||||||
|         mock__queue_consumer.assert_not_awaited() |         mock__queue_consumer.assert_not_awaited() | ||||||
|         mock_flag_set.assert_not_called() |         mock_flag_set.assert_not_called() | ||||||
|  |  | ||||||
|         mock_is_open.reset_mock() |         self.task_pool._locked = True | ||||||
|  |  | ||||||
|         mock_is_open.return_value = True |  | ||||||
|         self.assertIsNone(await self.task_pool._map(mock_func, args_iter, stars, num_tasks, end_cb, cancel_cb)) |         self.assertIsNone(await self.task_pool._map(mock_func, args_iter, stars, num_tasks, end_cb, cancel_cb)) | ||||||
|         mock__set_up_args_queue.assert_called_once_with(args_iter, num_tasks) |         mock__set_up_args_queue.assert_called_once_with(args_iter, num_tasks) | ||||||
|         mock__queue_consumer.assert_has_awaits(qsize * [call(mock_q, mock_flag, mock_func, arg_stars=stars, |         mock__queue_consumer.assert_has_awaits(qsize * [call(mock_q, mock_flag, mock_func, arg_stars=stars, | ||||||
|   | |||||||
							
								
								
									
										143
									
								
								tests/test_server.py
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										143
									
								
								tests/test_server.py
									
									
									
									
									
										Normal file
									
								
							| @@ -0,0 +1,143 @@ | |||||||
|  | import asyncio | ||||||
|  | import logging | ||||||
|  | from pathlib import Path | ||||||
|  | from unittest import IsolatedAsyncioTestCase | ||||||
|  | from unittest.mock import AsyncMock, MagicMock, patch | ||||||
|  |  | ||||||
|  | from asyncio_taskpool import server | ||||||
|  | from asyncio_taskpool.client import ControlClient, UnixControlClient | ||||||
|  |  | ||||||
|  |  | ||||||
|  | FOO, BAR = 'foo', 'bar' | ||||||
|  |  | ||||||
|  |  | ||||||
|  | class ControlServerTestCase(IsolatedAsyncioTestCase): | ||||||
|  |     log_lvl: int | ||||||
|  |  | ||||||
|  |     @classmethod | ||||||
|  |     def setUpClass(cls) -> None: | ||||||
|  |         cls.log_lvl = server.log.level | ||||||
|  |         server.log.setLevel(999) | ||||||
|  |  | ||||||
|  |     @classmethod | ||||||
|  |     def tearDownClass(cls) -> None: | ||||||
|  |         server.log.setLevel(cls.log_lvl) | ||||||
|  |  | ||||||
|  |     def setUp(self) -> None: | ||||||
|  |         self.abstract_patcher = patch('asyncio_taskpool.server.ControlServer.__abstractmethods__', set()) | ||||||
|  |         self.mock_abstract_methods = self.abstract_patcher.start() | ||||||
|  |         self.mock_pool = MagicMock() | ||||||
|  |         self.kwargs = {FOO: 123, BAR: 456} | ||||||
|  |         self.server = server.ControlServer(pool=self.mock_pool, **self.kwargs) | ||||||
|  |  | ||||||
|  |     def tearDown(self) -> None: | ||||||
|  |         self.abstract_patcher.stop() | ||||||
|  |  | ||||||
|  |     def test_client_class_name(self): | ||||||
|  |         self.assertEqual(ControlClient.__name__, server.ControlServer.client_class_name) | ||||||
|  |  | ||||||
|  |     async def test_abstract(self): | ||||||
|  |         with self.assertRaises(NotImplementedError): | ||||||
|  |             args = [AsyncMock()] | ||||||
|  |             await self.server._get_server_instance(*args) | ||||||
|  |         with self.assertRaises(NotImplementedError): | ||||||
|  |             self.server._final_callback() | ||||||
|  |  | ||||||
|  |     def test_init(self): | ||||||
|  |         self.assertEqual(self.mock_pool, self.server._pool) | ||||||
|  |         self.assertEqual(self.kwargs, self.server._server_kwargs) | ||||||
|  |         self.assertIsNone(self.server._server) | ||||||
|  |  | ||||||
|  |     def test_pool(self): | ||||||
|  |         self.assertEqual(self.mock_pool, self.server.pool) | ||||||
|  |  | ||||||
|  |     def test_is_serving(self): | ||||||
|  |         self.server._server = MagicMock(is_serving=MagicMock(return_value=FOO + BAR)) | ||||||
|  |         self.assertEqual(FOO + BAR, self.server.is_serving()) | ||||||
|  |  | ||||||
|  |     @patch.object(server, 'ControlSession') | ||||||
|  |     async def test__client_connected_cb(self, mock_client_session_cls: MagicMock): | ||||||
|  |         mock_client_handshake, mock_listen = AsyncMock(), AsyncMock() | ||||||
|  |         mock_client_session_cls.return_value = MagicMock(client_handshake=mock_client_handshake, listen=mock_listen) | ||||||
|  |         mock_reader, mock_writer = MagicMock(), MagicMock() | ||||||
|  |         self.assertIsNone(await self.server._client_connected_cb(mock_reader, mock_writer)) | ||||||
|  |         mock_client_session_cls.assert_called_once_with(self.server, mock_reader, mock_writer) | ||||||
|  |         mock_client_handshake.assert_awaited_once_with() | ||||||
|  |         mock_listen.assert_awaited_once_with() | ||||||
|  |  | ||||||
|  |     @patch.object(server.ControlServer, '_final_callback') | ||||||
|  |     async def test__serve_forever(self, mock__final_callback: MagicMock): | ||||||
|  |         mock_aenter, mock_serve_forever = AsyncMock(), AsyncMock(side_effect=asyncio.CancelledError) | ||||||
|  |         self.server._server = MagicMock(__aenter__=mock_aenter, serve_forever=mock_serve_forever) | ||||||
|  |         with self.assertLogs(server.log, logging.DEBUG): | ||||||
|  |             self.assertIsNone(await self.server._serve_forever()) | ||||||
|  |         mock_aenter.assert_awaited_once_with() | ||||||
|  |         mock_serve_forever.assert_awaited_once_with() | ||||||
|  |         mock__final_callback.assert_called_once_with() | ||||||
|  |  | ||||||
|  |         mock_aenter.reset_mock() | ||||||
|  |         mock_serve_forever.reset_mock(side_effect=True) | ||||||
|  |         mock__final_callback.reset_mock() | ||||||
|  |  | ||||||
|  |         self.assertIsNone(await self.server._serve_forever()) | ||||||
|  |         mock_aenter.assert_awaited_once_with() | ||||||
|  |         mock_serve_forever.assert_awaited_once_with() | ||||||
|  |         mock__final_callback.assert_called_once_with() | ||||||
|  |  | ||||||
|  |     @patch.object(server, 'create_task') | ||||||
|  |     @patch.object(server.ControlServer, '_serve_forever', new_callable=MagicMock()) | ||||||
|  |     @patch.object(server.ControlServer, '_get_server_instance') | ||||||
|  |     async def test_serve_forever(self, mock__get_server_instance: AsyncMock, mock__serve_forever: MagicMock, | ||||||
|  |                                  mock_create_task: MagicMock): | ||||||
|  |         mock__serve_forever.return_value = mock_awaitable = 'some_coroutine' | ||||||
|  |         mock_create_task.return_value = expected_output = 12345 | ||||||
|  |         output = await self.server.serve_forever() | ||||||
|  |         self.assertEqual(expected_output, output) | ||||||
|  |         mock__get_server_instance.assert_awaited_once_with(self.server._client_connected_cb, **self.kwargs) | ||||||
|  |         mock__serve_forever.assert_called_once_with() | ||||||
|  |         mock_create_task.assert_called_once_with(mock_awaitable) | ||||||
|  |  | ||||||
|  |  | ||||||
|  | class UnixControlServerTestCase(IsolatedAsyncioTestCase): | ||||||
|  |     log_lvl: int | ||||||
|  |  | ||||||
|  |     @classmethod | ||||||
|  |     def setUpClass(cls) -> None: | ||||||
|  |         cls.log_lvl = server.log.level | ||||||
|  |         server.log.setLevel(999) | ||||||
|  |  | ||||||
|  |     @classmethod | ||||||
|  |     def tearDownClass(cls) -> None: | ||||||
|  |         server.log.setLevel(cls.log_lvl) | ||||||
|  |  | ||||||
|  |     def setUp(self) -> None: | ||||||
|  |         self.base_init_patcher = patch.object(server.ControlServer, '__init__') | ||||||
|  |         self.mock_base_init = self.base_init_patcher.start() | ||||||
|  |         self.mock_pool = MagicMock() | ||||||
|  |         self.path = '/tmp/asyncio_taskpool' | ||||||
|  |         self.kwargs = {FOO: 123, BAR: 456} | ||||||
|  |         self.server = server.UnixControlServer(pool=self.mock_pool, path=self.path, **self.kwargs) | ||||||
|  |  | ||||||
|  |     def tearDown(self) -> None: | ||||||
|  |         self.base_init_patcher.stop() | ||||||
|  |  | ||||||
|  |     def test__client_class(self): | ||||||
|  |         self.assertEqual(UnixControlClient, self.server._client_class) | ||||||
|  |  | ||||||
|  |     def test_init(self): | ||||||
|  |         self.assertEqual(Path(self.path), self.server._socket_path) | ||||||
|  |         self.mock_base_init.assert_called_once_with(self.mock_pool, **self.kwargs) | ||||||
|  |  | ||||||
|  |     @patch.object(server, 'start_unix_server') | ||||||
|  |     async def test__get_server_instance(self, mock_start_unix_server: AsyncMock): | ||||||
|  |         mock_start_unix_server.return_value = expected_output = 'totally_a_server' | ||||||
|  |         mock_callback, mock_kwargs = MagicMock(), {'a': 1, 'b': 2} | ||||||
|  |         args = [mock_callback] | ||||||
|  |         output = await self.server._get_server_instance(*args, **mock_kwargs) | ||||||
|  |         self.assertEqual(expected_output, output) | ||||||
|  |         mock_start_unix_server.assert_called_once_with(mock_callback, Path(self.path), **mock_kwargs) | ||||||
|  |  | ||||||
|  |     def test__final_callback(self): | ||||||
|  |         self.server._socket_path = MagicMock() | ||||||
|  |         self.assertIsNone(self.server._final_callback()) | ||||||
|  |         self.server._socket_path.unlink.assert_called_once_with() | ||||||
| @@ -7,12 +7,12 @@ The minimum required setup is a "worker" coroutine function that can do somethin | |||||||
| The following demo code enables full log output first for additional clarity. It is complete and should work as is. | The following demo code enables full log output first for additional clarity. It is complete and should work as is. | ||||||
|  |  | ||||||
| ### Code | ### Code | ||||||
|  |  | ||||||
| ```python | ```python | ||||||
| import logging | import logging | ||||||
| import asyncio | import asyncio | ||||||
|  |  | ||||||
| from asyncio_taskpool.pool import SimpleTaskPool | from asyncio_taskpool import SimpleTaskPool | ||||||
|  |  | ||||||
|  |  | ||||||
| logging.getLogger().setLevel(logging.NOTSET) | logging.getLogger().setLevel(logging.NOTSET) | ||||||
| logging.getLogger('asyncio_taskpool').addHandler(logging.StreamHandler()) | logging.getLogger('asyncio_taskpool').addHandler(logging.StreamHandler()) | ||||||
| @@ -23,8 +23,8 @@ async def work(n: int) -> None: | |||||||
|     Pseudo-worker function.  |     Pseudo-worker function.  | ||||||
|     Counts up to an integer with a second of sleep before each iteration. |     Counts up to an integer with a second of sleep before each iteration. | ||||||
|     In a real-world use case, a worker function should probably have access  |     In a real-world use case, a worker function should probably have access  | ||||||
|     to some synchronisation primitive or shared resource to distribute work  |     to some synchronisation primitive (such as a queue) or shared resource | ||||||
|     between an arbitrary number of workers. |     to distribute work between an arbitrary number of workers. | ||||||
|     """ |     """ | ||||||
|     for i in range(n): |     for i in range(n): | ||||||
|         await asyncio.sleep(1) |         await asyncio.sleep(1) | ||||||
| @@ -38,7 +38,7 @@ async def main() -> None: | |||||||
|     await pool.start()  # launches work task 3 |     await pool.start()  # launches work task 3 | ||||||
|     await asyncio.sleep(1.5)  # lets the tasks work for a bit |     await asyncio.sleep(1.5)  # lets the tasks work for a bit | ||||||
|     pool.stop(2)  # cancels tasks 3 and 2 |     pool.stop(2)  # cancels tasks 3 and 2 | ||||||
|     pool.close()  # required for the last line |     pool.lock()  # required for the last line | ||||||
|     await pool.gather()  # awaits all tasks, then flushes the pool |     await pool.gather()  # awaits all tasks, then flushes the pool | ||||||
|  |  | ||||||
|  |  | ||||||
| @@ -60,7 +60,7 @@ did 1 | |||||||
| did 1 | did 1 | ||||||
| did 1 | did 1 | ||||||
| did 0 | did 0 | ||||||
| SimpleTaskPool-0 is closed! | SimpleTaskPool-0 is locked! | ||||||
| Cancelling SimpleTaskPool-0_Task-3 ... | Cancelling SimpleTaskPool-0_Task-3 ... | ||||||
| Cancelled SimpleTaskPool-0_Task-3 | Cancelled SimpleTaskPool-0_Task-3 | ||||||
| Ended SimpleTaskPool-0_Task-3 | Ended SimpleTaskPool-0_Task-3 | ||||||
| @@ -86,12 +86,12 @@ As with the simple example, we need "worker" coroutine functions that can do som | |||||||
| The following demo code enables full log output first for additional clarity. It is complete and should work as is. | The following demo code enables full log output first for additional clarity. It is complete and should work as is. | ||||||
|  |  | ||||||
| ### Code | ### Code | ||||||
|  |  | ||||||
| ```python | ```python | ||||||
| import logging | import logging | ||||||
| import asyncio | import asyncio | ||||||
|  |  | ||||||
| from asyncio_taskpool.pool import TaskPool | from asyncio_taskpool import TaskPool | ||||||
|  |  | ||||||
|  |  | ||||||
| logging.getLogger().setLevel(logging.NOTSET) | logging.getLogger().setLevel(logging.NOTSET) | ||||||
| logging.getLogger('asyncio_taskpool').addHandler(logging.StreamHandler()) | logging.getLogger('asyncio_taskpool').addHandler(logging.StreamHandler()) | ||||||
| @@ -126,15 +126,14 @@ async def main() -> None: | |||||||
|     # The second one will start (with ID 3), only once there is room in the pool, |     # The second one will start (with ID 3), only once there is room in the pool, | ||||||
|     # which -- in this example -- will be the case after ID 2 ends; |     # which -- in this example -- will be the case after ID 2 ends; | ||||||
|     # until then the `starmap` method call **will block**! |     # until then the `starmap` method call **will block**! | ||||||
|     # Once there is room in the pool again, the third one will start immediately (and receive ID 4). |     # Once there is room in the pool again, the third and fourth will each start (with IDs 4 and 5) | ||||||
|     # The last one will start (with ID 5) **only** once there is room in the pool **and** no more than one of these  |     # **only** once there is room in the pool **and** no more than one of these last four tasks is running. | ||||||
|     # last four tasks is running. |  | ||||||
|     args_list = [(0, 10), (10, 20), (20, 30), (30, 40)] |     args_list = [(0, 10), (10, 20), (20, 30), (30, 40)] | ||||||
|     print("Calling `starmap`...") |     print("Calling `starmap`...") | ||||||
|     await pool.starmap(other_work, args_list, num_tasks=2) |     await pool.starmap(other_work, args_list, num_tasks=2) | ||||||
|     print("`starmap` returned") |     print("`starmap` returned") | ||||||
|     # Now we close the pool, so that we can safely await all our tasks. |     # Now we lock the pool, so that we can safely await all our tasks. | ||||||
|     pool.close() |     pool.lock() | ||||||
|     # Finally, we block, until all tasks have ended. |     # Finally, we block, until all tasks have ended. | ||||||
|     print("Called `gather`") |     print("Called `gather`") | ||||||
|     await pool.gather() |     await pool.gather() | ||||||
| @@ -178,11 +177,11 @@ work with 150 | |||||||
| work with 150 | work with 150 | ||||||
| other_work with 8 | other_work with 8 | ||||||
| Ended TaskPool-0_Task-2    <--- here Task-2 makes room in the pool and unblocks `main()` | Ended TaskPool-0_Task-2    <--- here Task-2 makes room in the pool and unblocks `main()` | ||||||
| TaskPool-0 is closed! | TaskPool-0 is locked! | ||||||
| Started TaskPool-0_Task-3 | Started TaskPool-0_Task-3 | ||||||
| other_work with 9 | other_work with 9 | ||||||
| `starmap` returned | `starmap` returned | ||||||
| Called `gather` | Called `gather`    <--- now this will block `main()` until all tasks have ended | ||||||
| work with 160 | work with 160 | ||||||
| work with 160 | work with 160 | ||||||
| other_work with 10 | other_work with 10 | ||||||
| @@ -230,3 +229,5 @@ other_work with 39 | |||||||
| Done. | Done. | ||||||
| Ended TaskPool-0_Task-5 | Ended TaskPool-0_Task-5 | ||||||
| ``` | ``` | ||||||
|  |  | ||||||
|  | © 2022 Daniil Fajnberg | ||||||
|   | |||||||
| @@ -1,3 +1,25 @@ | |||||||
|  | __author__ = "Daniil Fajnberg" | ||||||
|  | __copyright__ = "Copyright © 2022 Daniil Fajnberg" | ||||||
|  | __license__ = """GNU LGPLv3.0 | ||||||
|  |  | ||||||
|  | This file is part of asyncio-taskpool. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is free software: you can redistribute it and/or modify it under the terms of | ||||||
|  | version 3.0 of the GNU Lesser General Public License as published by the Free Software Foundation. | ||||||
|  |  | ||||||
|  | asyncio-taskpool is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;  | ||||||
|  | without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  | ||||||
|  | See the GNU Lesser General Public License for more details. | ||||||
|  |  | ||||||
|  | You should have received a copy of the GNU Lesser General Public License along with asyncio-taskpool.  | ||||||
|  | If not, see <https://www.gnu.org/licenses/>.""" | ||||||
|  |  | ||||||
|  | __doc__ = """ | ||||||
|  | Working example of a UnixControlServer in combination with the SimpleTaskPool. | ||||||
|  | Use the main CLI client to interface at the socket. | ||||||
|  | """ | ||||||
|  |  | ||||||
|  |  | ||||||
| import asyncio | import asyncio | ||||||
| import logging | import logging | ||||||
|  |  | ||||||
| @@ -48,12 +70,12 @@ async def main() -> None: | |||||||
|     control_server_task = await UnixControlServer(pool, path='/tmp/py_asyncio_taskpool.sock').serve_forever() |     control_server_task = await UnixControlServer(pool, path='/tmp/py_asyncio_taskpool.sock').serve_forever() | ||||||
|     # We block until `.task_done()` has been called once by our workers for every item placed into the queue. |     # We block until `.task_done()` has been called once by our workers for every item placed into the queue. | ||||||
|     await q.join() |     await q.join() | ||||||
|     # Since we don't need any "work" done anymore, we can close our control server by cancelling the task. |     # Since we don't need any "work" done anymore, we can lock our control server by cancelling the task. | ||||||
|     control_server_task.cancel() |     control_server_task.cancel() | ||||||
|     # Since our workers should now be stuck waiting for more items to pick from the queue, but no items are left, |     # Since our workers should now be stuck waiting for more items to pick from the queue, but no items are left, | ||||||
|     # we can now safely cancel their tasks. |     # we can now safely cancel their tasks. | ||||||
|     pool.stop_all() |     pool.stop_all() | ||||||
|     pool.close() |     pool.lock() | ||||||
|     # Finally we allow for all tasks to do do their cleanup, if they need to do any, upon being cancelled. |     # Finally we allow for all tasks to do do their cleanup, if they need to do any, upon being cancelled. | ||||||
|     # We block until they all return or raise an exception, but since we are not interested in any of their exceptions, |     # We block until they all return or raise an exception, but since we are not interested in any of their exceptions, | ||||||
|     # we just silently collect their exceptions along with their return values. |     # we just silently collect their exceptions along with their return values. | ||||||
|   | |||||||
		Reference in New Issue
	
	Block a user