#+TITLE: Network Inventory :preamble: #+author: Andreas Zweili :end: * Introduction ** How to work with this file 1. All URLs should get included as footnotes on order to have a cleaner layout. 2. All tasks need a priority, tasks with a priority higher than C are considered must haves for the 1.0 release. Everything with C or lower is considered a nice to have an might never get implemented. 3. Once an item changes its status to either DONE or CANCELLED it gets moved to the "Done" heading. CANCELLED items should get an explanation why they were cancelled. ** Vocabulary - Technician :: An employee working with the inventory tool and full access to its information. - Customer :: A customer which owns some of the devices in the inventory tool and might have access to only the information related to the devices he owns. * TODO Must Have [0/14] ** TODO [#A] Forms [0/12] *** TODO Computer Forms [0/3] - [X] Add - [X] Update - [ ] Delete **** NEXT CPU Forms - [X] Add - [X] Update - [X] Delete **** NEXT RAM Forms - [X] Add - [X] Update - [X] Delete **** NEXT Disk Forms - [X] Add - [X] Update - [X] Delete *** NEXT Customer Forms - [X] Add - [ ] Update - [ ] Delete *** NEXT Backup Forms - [ ] Add - [ ] Update - [X] Delete *** NEXT UserLicense Forms - [ ] Add - [ ] Update - [ ] Delete *** NEXT DeviceLicense Forms - [X] Add - [X] Update - [X] Delete *** NEXT Net Forms - [ ] Add - [ ] Update - [ ] Delete *** NEXT Software Forms - [ ] Add - [ ] Update - [ ] Delete *** NEXT Users Forms - [ ] Add - [ ] Update - [ ] Delete *** NEXT Groups Forms - [ ] Add - [ ] Update - [ ] Delete *** NEXT Limit the dropdowns in update views When a user edits an object he should only be able to select objects from the dropdowns he's allowed to view. Meaning that we have to limit every dropdown in an update view to the customer he's allowed to see. Like only the Nets or Users related to that customer. We can use the get_objects helper function in core.utils. *** NEXT Limit the customer views to superusers *** NEXT Add + button for nets in the DeviceInNet form This way we can quickly add a net when we want to add an IP to a device. ** NEXT [#A] a backup should be able to contain multiple computers :model: ** NEXT [#A] implement view for groups ** NEXT [#A] Add a feature to copy objects This is a must have for the first release. As a user I would like to have a way of quickly copy an object and make some adjustments in order to add many objects after another. ** NEXT [#A] Add search boxes to the views. So that one can search for a string in the responding column. ** NEXT [#A] Create custom user model :model: It is best practice to create a custom user model to allow future modifications to the users without causing problems. ** NEXT [#B] Implement a license check into all forms This should prevent technicians from assigning licenses which the customer has already fully used. ** NEXT [#B] calculate licence usage for customer The view should show the licenses which the customer currently has available and how many are already used. In addition it should show a visual warning to the technician when the limit is reached. ** NEXT [#B] Convert the NETSheet Data file to YML fixtures. A lot of this is already done. Only the hardware models are currently missing. ** NEXT [#B] Have a look at the documentation of django-nested-admin I implemented nested-admin currently in a very basic way. I should read the documentation in order to make sure that I'm using it correctly. ** NEXT [#B] have a look at django select_related, it might solve a problem for me. I often find myself trying to get related objects. The method select_related might help with that [fn:1]: - ** NEXT [#B] Extend the Admin tables The admin tables show currently very little information about the various objects. At minimum every object should display the customer it belongs to. ** NEXT [#B] In the warranty form validate the date inputs The starting date shouldn't be allowed to be newer than the end date. ** NEXT [#C] Filter Hardware Model to corresponding device manufacturer When changing the HardwareModel field of a device the dropdown should be filtered to the provided DeviceManufacturer. Currently it could still make sense to make the DeviceManufacturer only available through the HardwareModel. This way we wouldn't have to filter the HardwareModel dropdown. However we would loose the ability to only select the DeviceManufacturer for a device in case we don't know the specific model which happens quite often. * TODO Nice to Have [0/11] ** NEXT [#C] allow technicians to add custom fields This would allow technicians to create custom models without change Maybe this approach would be something [fn:2]: ** NEXT [#C] calculate the used space on a host Means calculate the size all the VMs would use if they were thick. This could help a technician to properly plan ressources on a host. ** NEXT [#C] include a RAID calculator I would like to use this to show the usable space in a RAID system. Currently we enter this information by hand but it would be easier to calculate it automatically [fn:3]. - ** NEXT [#C] Get warranty information from Dell We sell a lot of Dell devices and it would be nice to use the service tags to collect the warranty information directly from Dell. There's an API for that [fn:4]: ** NEXT [#C] A "to deactivate" feature on inventory users This way a technician could mark a user for deactivation and anyone could check if there are users to deactivate. This would help if we would've to deactivate a user at a certain date. The inventory tool could then show to all technicians that the user needs to be deactivated. Then any technician could deactivate the user and not just the technician responsible for the customer, increasing the security of the customer. ** NEXT [#C] change the admin url For security reasons it's recommended to change the name of the admin panel url. This way automated tools can't find it so easy. It only increases the security slightly. ** NEXT [#C] implement guardian This needs to be done for basically every model which lives on a view. E.g. ~BackupListView~, ~SoftwareListView~. I can’t remember how this should be implemented. However it might be implemented in the customer table view. The security concept works like this: 1. check if the user is logged in 2. check if the user is allowed to view the customer, if not return an error 3. Get all matching objects which the user is allowed to view. Step two can't be replaced by an empty table because we need a customer object to operate on. Therefore it's better to quickly check the customer before we fetch all the other objects from the database. ** NEXT [#C] Implement an excel import and export might be achieved with the django-excel project [fn:6]. This might be a nice to have feature but the copy button is more important to achieve a similar funcition. ** NEXT [#C] implement SoftwareDetailView I don't remember what the initial idea here was. We could show here which customers are using this software. But that is currently a really low priority item. ** NEXT [#C] Implement the .all command in templates This stackoverflow post should help [fn:9] ** NEXT [#B] Check tests for response.context[‚table‘] This would allow for tests of the views which check explicitly what gets returned by the view. Might be easier/faster then rendering the whole view. However for some views it would be better to test the final view because the template contains logic which can fail. * Resources [fn:11] https://gist.github.com/neara/6209563 [fn:10] https://github.com/timhughes/django-cbv-inline-formset [fn:9] https://stackoverflow.com/questions/4270330/django-show-a-manytomanyfield-in-a-template [fn:8] https://docs.djangoproject.com/en/3.0/ref/forms/fields/#multiplechoicefield [fn:7] https://stackoverflow.com/questions/23059088/manytomany-field-check-if-relation-exists [fn:6] https://github.com/pyexcel-webwares/django-excel [fn:5] https://docs.djangoproject.com/en/2.2/topics/http/urls/#views-extra-options [fn:4] https://www.programmableweb.com/api/dell-warranty-status-rest-api [fn:3] https://thoughtworksnc.com/2017/08/30/writing-a-raid-calculator-in-python [fn:2] https://stackoverflow.com/questions/34907014/django-allow-user-to-add-fields-to-model [fn:1] https://docs.djangoproject.com/en/2.2/ref/models/querysets/#select-related ** Class Based Views - http://ccbv.co.uk/ ** Design *** Admin themes - django-grappelli - django-suit - django-admin-bootstrapped ** Forms - https://django-crispy-forms.readthedocs.io/en/latest/index.html - https://stackoverflow.com/questions/25321423/django-create-inline-forms-similar-to-django-admin*25340256 - https://stackoverflow.com/questions/5171365/django-inline-form-with-custom-forms ** Permissions - https://django-guardian.readthedocs.io/en/stable/userguide/assign.html - https://github.com/dfunckt/django-rules/blob/master/README.rst #+begin_src python decororator (function) : if user has permission(object.customer): return function #+end_src Maybe it would be possible to add a property to the classes which allows to access the customer of an object like this: #+begin_src python object.customer #+end_src * Links to include - https://docs.djangoproject.com/en/2.2/ref/models/querysets/#id4 - https://docs.djangoproject.com/en/2.2/ref/request-response/ - https://duckduckgo.com/?q=django+get_related&t=fpas&ia=qa - https://pybit.es/selenium-pytest-and-django.html - https://stackoverflow.com/questions/28533174/programatically-accessing-django-models-from-another-app - https://stackoverflow.com/questions/54592026/how-to-create-a-custom-mixin-in-django - https://stackoverflow.com/questions/58307055/access-django-model-name-from-admin-url-pattern - https://stackoverflow.com/questions/6069070/how-to-use-permission-required-decorators-on-django-class-based-views#6069444 * Done ** DONE Recreate the RM in draw.io The Dia RM is okay but not really that great. Draw.io would give a better result. ** DONE create multiple requirements files ** DONE put passwords into environment variables ** DONE Permissions recherchieren ** DONE customer tabelle erweitern mit listen ** DONE Models erstellen ** DONE Add a Counter to the RAM Modules ** DONE Create a NET category where a device can live in. This NET Category should display it's IP range, Subnet mask and show it's DHCP range if one is configured. ** DONE Create class DeviceInNet This class shows the relationship between the device and a NET. An attribute of a DeviceInNet should be an IP address. ** DONE Create an abstract company class ** DONE Create Customer and a Manufacturer sub class Those two would be based on the company class. I'm currently not sure how I should handle the case where a company is both a customer and a manufacturer. ** DONE A text field next to the customer where one can enter additional information which can't be put into the normal documentation. ** DONE Fix test for net detail view ** DONE NETs, add a description field, for NETs like HEHImmo it might be nice to have a short description for what it is intendet. ** DONE ComputerDetailView, add link to SoftwareDetailView ** DONE implement NETSheet list this view should give an overview of all the devices in the NET and there current IP Address. ** DONE implement BackupListView ** DONE Filter the queryset in the AllComputerView so that it only shows the customers the current user is allowed to view ** DONE Disks in RAID and RAID have overlapping Felds (disks appear on both). And they don't have the proper relationship. There can be disks from variing sizes in a RAID therefore the relationship between DisksInRaid and RaidInComputer needs to be a manytoone relationship ** DONE fix column name links in customer table they throw an error when one clicks on them. ** DONE ComputerDetailView, add all properties to the view table ** DONE implement UserListView ** DONE implement SoftwareListView this and the next view would probably better be a License view. Since the software should be available to all devices from all customers. It doesn’t make much sense to add 100 of different Office softwares. Probably a Software model could be attached to a License model. ** DONE implement UserDetailView ** DONE Implement the license so that it can get attached to a user when the user gets created. This way they might get less easily forgotten. ** DONE fix the Makefile so that the fixtures don't get applies twice. This is already done for the ~make local~ command but needs fixing in the ~make~ command. However there's a bit more difficult because it runs in Docker and with PostgreSQL ** DONE refactor the project to have a core app. CLOSED: [2020-01-14 Tue 21:25] This way I can split the project into multiple apps such as Customer, Computer, Backups etc. and import the shared models from core. This allows me to split the views and tests over multiple apps making the whole thing a bit easier to understand. See the Notability note for more information. https://github.com/netbox-community/netbox/tree/develop/netbox might provide an example When doing the refactor I should correct the imports. The current system is very annoying when I add a new object/class. ** DONE Hardware Model CLOSED: [2020-02-14 Fri 20:28] I'm currently unsure if I should implement a hardware model. With this model I could add the hardware model to a device. Currently this capability is missing. ** DONE add a list of assigned users and computers to the license view CLOSED: [2020-02-15 Sat 18:53] ** DONE Server mit NGINX aufsetzen CLOSED: [2020-02-15 Sat 18:56] - https://docs.djangoproject.com/en/2.2/howto/deployment/wsgi/uwsgi/ - https://uwsgi-docs.readthedocs.io/en/latest/tutorials/Django_and_nginx.html - https://linuxconfig.org/how-to-host-django-with-nginx-on-ubuntu-18-04-bionic-beaver-linux ** DONE CustomerListView [3/3] CLOSED: [2020-02-16 Sun 18:45] add all the objects - [X] Backup - [X] Software - [X] Users ** DONE implement permission decorators currently all the items can get viewed. CLOSED: [2020-02-16 Sun 18:52] I either have to implement a decorator for each object type or find a general way. This problem is only related to detail views. The tables and lists have a general way to check the permission. I maybe could get the model name from the url, this Stackoverflow post might help: - https://stackoverflow.com/questions/58307055/access-django-model-name-from-admin-url-pattern and get the object with this function: - https://stackoverflow.com/questions/28533174/programatically-accessing-django-models-from-another-app ** DONE Add tests for multiple nets and devices CLOSED: [2020-02-16 Sun 18:52] ** DONE rename variables for the querysets to XXXRelations CLOSED: [2020-02-16 Sun 18:53] ** DONE limit the queryset in the customer_table CLOSED: [2020-02-16 Sun 19:13] The queryset should only contain results which a users is allowed to see. ** DONE make sure the licenses models are correct. CLOSED: [2020-02-16 Sun 19:31] I think manytomany might not be the correct relation since a user should only be attached once to a user license and a computer should only be attached once to a computer license. However a user can stil have many licenses and a license can still have many users. ** DONE tables problem CLOSED: [2020-04-20 Mo 12:49] This isn't fully working yet in django_tables2 https://github.com/jieter/django-tables2/issues/721 #+begin_src diff -from django_tables2.utils import A + class CustomersTable(tables.Table): - name = tables.LinkColumn('customer', args=[A('pk')]) - nets = tables.LinkColumn('nets', text='Nets', args=[A('pk')]) - computers = tables.LinkColumn('computers', text='Computers', args=[A('pk')]) - devices = tables.LinkColumn('devices', text='Devices', args=[A('pk')]) - backups = tables.LinkColumn('backups', text='Backups', args=[A('pk')]) + name = tables.Column(linkify=("customer", [tables.A("pk")])) + nets = tables.Column(verbose_name="Nets", + linkify=("nets", [tables.A("pk")])) + computers = tables.Column(verbose_name="Computers", + linkify=("computers", [tables.A("pk")])) + devices = tables.Column(verbose_name="Devices", + linkify=("devices", [tables.A("pk")])) + backups = tables.Column(verbose_name="Backups", + linkify=dict(viewname="backups", args=[tables.A("pk")])) #+end_src ** DONE implement a warranty overview CLOSED: [2020-04-20 Mo 13:31] 2020-04-20 This is implementend now. I've setup the view so that customers can use the view as well and the content gets limited to what they are allowed to see. This view would show all devices which are running out of warranty, maybe this could be shown as well in the CustomerDetailView. So that we would've a list for the customers to see and one large list which shows the warranties for all customers for internal usuage. ** DONE Fix the warranty in the device view CLOSED: [2020-04-27 Mo 21:31] With currently the containers don't disappear fully. Should be easy to fix. ** CANCELLED Move the lists to their own page CLOSED: [2020-04-27 Mo 22:15] Since I have more devices than I thought it would provide a better overview than one big list. Forgot again what this exactly means. ** DONE [#A] cpu reduce required fields :model: CLOSED: [2020-04-30 Do 14:54] The CPU has many required fields but sometimes they don't make sense. For example when we have a virtual CPU we usually don't need to know the specific frequenzy. ** DONE [#A] the computer is missing a GPU :model: CLOSED: [2020-04-30 Do 15:45] CAD computers often have sppecial graphics cards which we should be able to track. ** DONE [#A] Days need to be manytomany :model: CLOSED: [2020-04-30 Do 19:21] A backup can be run on multiple days ** DONE [#A] the IP needs to be able to be null :model: CLOSED: [2020-04-30 Do 20:33] Currently it's always required however when a device is in DHCP mode we can't know the IP for sure. ** DONE [#B] Add the warranty to the Device, ConnectedDevice and Computer admin pages CLOSED: [2020-05-03 So 19:32] This is a relationship which a technician should be able to add directly on the device and not have to navigate to the warranty option first. ** DONE [#A] check if the connected device and device model could get merged :model: CLOSED: [2020-05-03 So 22:19] When using the forms for warranties and net relations they both point to different models but use the same logic so they always go back to the device details. However it's nice to differentiate between the connected device and the device in the forms. So that one only can select the connected devices in the dropdown. However the line between a connected device and a device is very thin. ** DONE Device Forms [2/2] CLOSED: [2020-05-03 So 22:20] - [X] Add - [X] Update - [X] Delete *** DONE Warranty Forms CLOSED: [2020-05-03 So 22:19] - [X] Add - [X] Update - [X] Delete *** DONE DeviceInNet Forms CLOSED: [2020-05-03 So 22:19] - [X] Add - [X] Update - [X] Delete ** DONE [#A] add a button to add a relation CLOSED: [2020-07-05 So 13:05] For the many to many relationships it might be a better idea to add them in a separate form. So that the view displays the information but when you want to add a new CPU for example it shows a button "Add CPU" which you can click an it brings you to a new page where the PC is already preselected and the CPU can get selected from a drop down. The update form currently adds a new relation every time I update. The idea above would be an easy solution. Another option to try out might be the MultipleChoice field[fn:8]. Maybe with this I can limit the number of times a relation can be added. ** DONE [#A] Fix the table for the connected devices CLOSED: [2020-07-05 So 13:36] Currently there is a column for "Device_ptr" I have no idea where that came from. ** DONE [#A] Add a view for DeviceManufacturer details CLOSED: [2020-07-05 So 13:39] since the DeviceManufacturer model now contains various contact details we should provide a details view to let uses access those details. ** DONE [#B] Show all models in the admin interface CLOSED: [2020-07-05 So 13:49] :LOGBOOK: - State "WAITING" from "NEXT" [2020-07-05 So 13:49] :END: Currently I'm hiding a lot of models from the admin interace to make it look a bit nicer. However for the production site we want to work mostly on the frontend. The admin page should then become really what it is. An admin inteface so that we can delete or add models which currently don't have a frontend interface. ** DONE [#C] Extend the CSS CLOSED: [2020-07-05 So 13:50] - A more centered layout would be nice - Maybe some colours ** CANCELLED [#C] update the url code [fn:5] CLOSED: [2020-07-05 So 13:56] I'm currently not sure what I wanted to do with this. ** CANCELLED [#C] Add a check to see if a software has a license attached to it. CLOSED: [2020-07-05 So 14:04] I forgot the reason why I need this. Maybe to show in general if a software has any licenses attached to it. Add a check to see if a software has a license attached to it.If so it increases the used licenses counter. Maybe with this this stackoverflow post can help [fn:7]. ** CANCELLED [#C] Inline Formfields like in the admin interface CLOSED: [2020-07-05 So 14:06] If already found various Github projects which might serves as an example[fn:10] [fn:11] currently not needed anymore I solve it with multiple form views because the manytomany always generate a new object when updating.